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Executive Summary 


Introduction 

The American Recovery and Reinvestment Act (ARRA or the Recovery Act) of 2009 provided an 
unprecedented level of funding for K-12 education. The program created a "historic opportunity to save 
hundreds of thousands of jobs, support states and school districts, and advance reforms and 
improvements that will create long-lasting results for our students and our nation." 1 Specifically, the 
Recovery Act allocated $70.6 billion in funding for K-12 education, of which $68.8 billion was awarded to 
states through a combination of newly created and existing grant programs, including the State Fiscal 
Stabilization Fund (SFSF) formula grants, Race to the Top (RTT) discretionary grants, and additional 
funding for the School Improvement Grant (SIG) program. As a way of promoting educational 
improvement, the Recovery Act required state recipients of ARRA funds to commit to advancing four 
specific education reform priorities, or "assurance areas": 

• "Making progress toward rigorous college- and career-ready standards and high-quality 
assessments that are valid and reliable for all students, including English language learners and 
students with disabilities, 

• Establishing pre-K to college and career data systems that track progress and foster continuous 
improvement, 

• Making improvements in teacher effectiveness and the equitable distribution of qualified 
teachers for all students, particularly students who are most in need, and 

• Providing intensive support and effective interventions to the lowest-performing schools." 2 

The four assurance areas were intended by the act's drafters to constitute an integrated, 
comprehensive vision of educational improvement that would be capable of raising the academic 
performance of all public school students. The vision embodied in the assurance areas begins with high 
expectations and accountability for student achievement (i.e., shared, rigorous standards and 
appropriate assessments). To support this vision, the Recovery Act's programs encouraged the 
development of data tools that can provide information to both help educators identify needs for 
improvement and provide feedback on the impact of educational changes on student learning. The act 
also recognizes the importance of effective educators by emphasizing improvements in their 
preparation, evaluation, and compensation and on achieving a more equitable distribution of effective 
teachers across schools within local education agencies (LEAs). Finally, to upgrade persistently low- 
performing schools, the act's programs provide incentives and tools for intervening in and improving 
these schools. By linking a commitment to the four assurance areas with receipt of funding, the 
Recovery Act signaled federal priorities; provided states, districts, and schools with incentives to initiate 
or intensify reforms in each of these areas; and encouraged states to pursue a combination of mutually 
supporting reform strategies. 


1 The American Recovery and Reinvestment Act of 2009: Saving and Creating Jobs and Reforming Education (March 2009). 
http://www2.ed.gov/policv/gen/leg/recoverv/implementation.html 

Ibid. 


2 


This is the final report of a multi-year U.S. Department of Education (ED) evaluation Charting the 
Progress of Education Reform: An Evaluation of the Recovery Act's Role. ED sought to track the adoption 
of policies and the implementation of practices associated with the education reforms promoted by the 
Recovery Act at the state, district, and school levels. This report focuses on the status of implementation 
of these reforms two full school years after the award of all Recovery Act funds (2011-12). It also 
describes the progress in implementing these reforms since these funds were first being distributed 
(2009-10). The report provides a snapshot of the status of reforms at particular points in time. The study 
did not collect information about the reasons why states, districts, and schools adopted or implemented 
these policies and practices. In addition, the report does not assess the relationship between the 
amount of Recovery Act funds received or the receipt of funds from specific grant programs and the 
implementation of reform. 3 Finally, the report does not address whether state, district, or school 
implementation of more reforms constitutes an integrated, comprehensive reform approach. 


Key Findings 


The indicators of reform tracked by the study reflect ED's priorities and key reform strategies 
within each of the four assurance areas as identified in its grant notices, regulations, and guidance for 
the Recovery Act programs. 4 The indicators were designed to provide a high-level snapshot of whether 
SEAs, districts, or schools had a particular policy in place, provided support, or carried out a particular 
activity. 


Among the indicators of reform that the study tracked, the prevalence and progress of reform 
implementation from 2009-10 to 2011-12 varied by assurance area and level (state, district, or school). 
At the state and school level, implementation of reforms increased in multiple assurance areas while at 
the district level only one assurance area showed increased activity. Across all levels, progress was seen 
most often in the standards and assessment reform indicators and least often for indicators of educator 
effectiveness and workforce development. 

• At the state level, progress was made on 16 of the 18 indicators that the study tracked. The two 
where progress was not found both related to the capabilities of state level data systems (figures 
ES-1, ES-4, ES-7, ES-10). The number of state education agencies (SEAs) that reported supporting use 
of student achievement gains for principal evaluation went from 6 in 2009-10 to 22 in 2011-12, the 
largest increase found. However, in 2011-12 more SEAs were carrying out the four reforms related 
to standards and assessments (between 42 and 46) than were carrying out the three related to 
educator evaluation and compensation (between 14 and 22). The largest increase in the area of 
standards and assessments was in providing instructional materials or curriculum assistance, where 
the number of SEAs increased from 29 in 2010-11 to 42 in 2011-12. 


3 All states committed to the four assurance areas as a condition for receiving funds, and more than 90 percent of the Recovery 
Act funding was awarded by formula to all states. At the district level, it is difficult to accurately account for all funding (or 
benefits) received by districts from all Recovery Act programs. For example, the subgrant data in Recovery.gov does not 
systematically include subgrants of less than $25,000. Additionally, some reforms depend on, or are facilitated by, state 
actions (and likely funded by dollars retained at the state level). An analysis of district funding and reform implementation 
would miss this effect. 

4 We drew on the specific strategies and activities described in SFSF assurance indicators and descriptors, the RTT selection 
criteria, and guidance for the Title l-ARRA and SIG programs. 
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• Local-level findings for standards and assessments were in line with state-level findings. Progress 
was reported for three of the four district level indicators (figure ES-2) and all three of the school 
level indicators (figure ES-3). 

o In 2011-12, 73 percent of districts in states that adopted the Common Core State Standards 
(CCSS) reported providing professional development on new or revised state standards up 
from 59 percent in 2009-10, the largest increase found in a district level reform effort. Other 
areas where district reports indicate progress were increased awareness of state adoption 
of the CCSS and provision of standards-aligned instructional materials or curricular guidance. 

o Likewise, 78 percent of schools in CCSS states reported that teachers received standards- 
related professional development in 2011-12, up from 63 percent in 2009-10. In 2011-12, 
schools also were more likely to report that teachers received professional development 
targeted to helping English learners or students with disabilities master new standards or to 
use curriculum or materials aligned with new standards. 

• Findings related to educator effectiveness varied across the three levels. For example, the 
percentage of districts that reported using student achievement gains to make decisions about 
teacher tenure, dismissal, or assignment decreased over time (24 percent in 2011-12 compared with 
38 percent in 2009-10). However, there was an increase in the percentage of schools that reported 
using the same practices (34 percent in 2011-12 compared with 29 percent in 2009-10). Similarly, 
there was a small but significant decrease in the percentage of districts that reported they 
differentiated teacher compensation based on student achievement gains (8 percent in 2011-12 
compared with 12 percent in 2009-10), while there was no significant change at the school level (10 
percent in 2011-12 and 12 percent in 2009-10) (figures ES-8 and ES-9). 

• In the area of improving low-performing schools, the number of SEAs providing guidance on 
choosing and implementing SIG school intervention models increased to include all 51 SEAs in 2011- 
12. However, reports from districts with low-performing schools and from low-performing schools 
suggest that relatively little use was made of key elements of the SIG closure, restart, and 
turnaround models. For example, in 2011-12, 2 percent of districts with low-performing schools 
targeted these schools for closure, and 11 percent of low-performing schools reported that they 
replaced a substantial portion of teachers as part of school restructuring (figures ES-11 and ES-12). 

• As expected, states that received RTT awards were more likely than other states to implement 
reforms across the assurance areas in 2011-12. In keeping with expectations, large districts were 
significantly more likely than small and medium size districts to implement reforms across the 
assurance areas. 5 However, in contrast to expectations, generally no significant differences were 
found between high-poverty and other districts in the implementation of reforms. Low-performing 
schools were significantly less likely than other schools to report implementing two of the three 
reforms examined related to standards and assessments, but were significantly more likely than 
other schools to report using longitudinal data to track student achievement gains and use these 
gains to evaluate principals and to differentiate teacher compensation. 


5 As a reminder, the district-level analysis of reforms to support improvement in low-performing schools was limited to districts 
with low-performing schools. We did not analyze the data for districts with low-performing schools by district size and district 
poverty status. 
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• The most frequently reported major challenges in 2011-12 were in the area of educator workforce 
development (table ES-1). Among SEAs, 76 percent (35 of 46 rating the challenge) reported difficulty 
measuring student growth for teachers of subjects with no standardized tests as a major challenge. 
Large percentages of districts and schools rated insufficient funding to provide performance-based 
compensation or differentiated compensation as major challenges. 


Study Questions and Methods 


This study was designed to capture the implementation of the reform agenda promoted by the 
Recovery Act for each of three school years. Activity during the 2009-10 school year provided a baseline 
for the study, capturing the reforms already in place when Recovery Act funding was first being 
distributed. The study also examines progress of reform as measured by the total number of SEAs or 
percentage of districts and schools implementing reforms from one year to the next. Though states (and 
their SEAs) are the primary focus of the four assurance areas and the primary conduits for the act's 
financial assistance, understanding the progress of the reforms promoted by the act, also requires 
examining reform activity at the district and school levels. This is important because (1) states differ in 
the degree to which state policy determines district and school action, (2) state policy regulates or 
enables activities or programs for which districts or schools are responsible, and (3) a few of the act's 
component programs bypassed the state level to provide funds directly to districts promising to 
undertake specific reforms that were related to the assurance areas. It should be noted that while this 
report describes state-, district-, and school-level reform implementation it does not attempt to 
attribute change— or lack of change— to the Recovery Act's requirements or incentives. 

Study Questions 

Specifically, this report addresses the following questions: 

• To what extent did SEAs, districts, and schools report implementing key reform strategies 
promoted by the Recovery Act in the 2011-12 school year? 

• How much of the 2011-12 school year implementation reflects progress since the Recovery Act? 

• Did the extent of reform in the 2011-12 school year vary by relevant state, district, and school 
characteristics? 

• What were the greatest reform implementation challenges for SEAs, districts, and schools in the 
2011-12 school year? 

Data Sources 

The findings in this report were drawn primarily from surveys administered to all 50 SEAs and 
the District of Columbia (DC) and nationally representative samples of school districts and schools during 
spring 2011 and spring 2012. The spring 2011 surveys asked about SEA, district, and school activities in 
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the 2009-10 and 2010-11 school years. The spring 2012 surveys asked about SEA, district, and school 
activities in the 2011-12 school year. 6 

The study's findings also drew on two existing sources of data for state-level reform activity The 
National Alliance of Public Charter School's annual review of state charter school laws provided 
information on removing or reducing limits on charter schools. States' annual performance data for the 
SFSF program provided information about state education data systems. 

Indicators of Reform 

To address the study questions, we developed indicators of reform implementation at the state 
(18 indicators), district (21), and school (22) levels. These indicators, presented in the figures in the 
findings section below, reflect ED's priorities and key reform strategies within each of the four assurance 
areas as identified in its grant notices, regulations, and guidance for the Recovery Act programs. 7 The 
indicators were designed to provide a high-level snapshot of whether SEAs, districts, or schools had a 
particular policy in place, provided support, or carried out a particular activity. In general, the SEA 
indicators focus on state adoption of specific education policies and the states' role in supporting 
implementation of these reforms. The district indicators capture information about district adoption of 
specific education policies and the use of strategies to support and promote reform policies at the 
school level. The school indicators focus on whether specific practices or strategies associated with 
implementing state or district policies or programs related to the assurance areas were being used in 
schools. 


Because of the variety of potential SEA and district responses to Recovery Act reform 
requirements and because assurance areas could be met by using different approaches, the indicators 
often captured several ways in which a state, district, or school might implement a reform. Where 
appropriate, the indicators include multiple strategies, but do not assume that one approach is 
preferable to another. For these indicators, an SEA, district, or school was said to have met an indicator 
if it reported implementing any one of a particular set of related strategies. Some Recovery Act 
programs, however, have more prescriptive requirements. In these cases, states and districts had to 
take specified actions, such as adoption of the Common Core State Standards (CCSS) or the inclusion of 
student growth measures in educator evaluation systems, to meet an indicator. For these indicators, an 
SEA or district met the indicator only if a specific set of requirements were met. 

The full report provides the context and rationale for each indicator, and the appendices include 
a detailed description of components, decision rules, and specific Recovery Act requirements embodied 
in each indicator. Several factors affect interpretation of indicator results. First, the survey asked SEAs, 
districts, and schools to self-report on their reform activity. Some respondents may have over- or under- 
stated their status in implementing reforms promoted by the act. Second, the surveys relied on closed- 
ended questions to ask about particular reform policies, programs, strategies, and practices. It is 
possible that SEAs, districts, and schools may have been working on a reform in a way not captured by 


6 None of the surveys asked about reforms prior to 2009-10 and did not ask how long any particular reform had been 
underway. 

7 We drew on the specific strategies and activities described in SFSF assurance indicators and descriptors, the RTT selection 
criteria, and guidance for the Title l-ARRA and SIG programs. 
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our survey questions. Third, the indicators represent a high-level snapshot of SEA, district, and school 
response to Recovery Act reform priorities and do not measure the intensity or quality of reform efforts. 

Challenges 

In addition to measuring the extent of reform activity, the study examined challenges that SEAs, 
districts, and schools reported when implementing reforms in 2011-12. The surveys asked states, 
districts, and schools that were implementing any reforms in an assurance area whether they 
encountered particular challenges and whether each challenge was a major or minor one. 


Findings 


The findings below are organized by reform assurance area: standards and assessment, data 
systems, educator workforce development, and improving low-performing schools. Within each 
assurance area, we discuss the status of SEAs, districts, and schools on the Recovery Act reform agenda 
in 2011-12 and change since 2009-10 to get a measure of new activity or progress over time. 8 When 
discussing the results for progress over time at the district and school levels, we focus on findings where 
statistically significant increases or decreases were found. We conclude by presenting a summary of the 
top major challenges when working to implement reforms in each assurance area reported by SEAs, 
districts, and schools in 2011-12. 

The full report includes findings for the status of reform implementation in 2011-12 by several 
SEA, district, and school characteristics to determine if certain types of entities were more likely to 
report implementing reforms than others. At the state level, we examined reform implementation by 
whether the state received a first or second round RTT award. At the district level, we examined reform 
implementation by district poverty status and district size. For three of the four assurance areas, we also 
examined differences in reform implementation between schools identified as low performing and 
schools not so identified. The full report also identifies additional challenges (beyond those in the 
Executive Summary) that were reported as major in 2011-12, but were not among the top-rated major 
challenges. The final chapter of the report provides an overview of state, district, and school reform 
implementation across the four assurance areas. 

Standards and Assessments 

The Recovery Act cited the state-level adoption and implementation of rigorous college- and 
career-ready standards and aligned high-quality assessments as pivotal steps in accelerating educational 
improvement throughout the United States. As a condition for receipt of SFSF funds, the Recovery Act 
held states accountable for improving state academic standards and enhancing the quality of academic 
assessments. The final SFSF rules encouraged "states to work together to develop and implement 
common, internationally benchmarked standards and assessments aligned to those standards, in order 


s Note that some reform activities measured in the SEA indicators have the potential to continue over multiple years once they 
are instituted. For example, when a state issues standards or guidelines for teacher preparation programs, these are likely to 
remain in effect until new or revised standards or guidelines are issued. So, this one time action of issuing standards can 
represent a support for a reform effort that is ongoing or continued in future years. In measuring SEA reform activity for 
selected indicators, if a state reported activity in a previous year, the SEA is counted as meeting the indicator in subsequent 
years. 
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to ensure that students are college- and career-ready." While the act did not explicitly endorse the CCSS, 
new policies established incentives for states to adopt the CCSS in reading/English language arts and 
mathematics. Adoption of common standards also was intended to facilitate states working together to 
develop common assessments and to update those standards as needed over time, saving states time 
and money and reducing redundancy and inconsistencies across states. Under the act, ED funded two 
multi-state consortia, the Partnership for Assessment of Readiness for College and Careers (PARCC) and 
the Smarter Balanced Assessment Consortium (SBAC), to develop the next generation of K-12 
assessments for the general student population. 

The Investing in Innovation (i3) and RTT programs provided incentives for school districts and 
others to support the transition to the new standards and aligned assessments. One of the priority areas 
for the i3 grants was to fund innovations that complement the implementation of high standards and 
quality assessments. As part of the RTT grant selection criteria, participating districts were to collaborate 
with their SEAs to develop a high-quality plan to support the transition to college- and career-ready 
standards and aligned assessments. Figures ES-1 through ES-3 below summarize state, district, and 
school findings for the study's reform indicators for standards and assessments. Note that figures ES-2 
and ES-3 are limited to districts and schools in states that adopted the CCSS in mathematics and 
reading/English language arts. 

Implementation and Progress from 2010-11 to 2011-12 

State adoption of the CCSS and membership in a federally funded consortium developing 
aligned assessments increased between 2010-11 and 2011-12. Although most states had already 
adopted the CCSS by 2010-11, the number of SEAs reporting that they adopted the CCSS in mathematics 
and reading/English language arts increased from 43 to 46 SEAs in 2011-12 (figure ES-1). 9 In addition, 
the number of SEAs reporting that they were members of a federally funded consortium developing 
assessments aligned to the CCSS increased from 43 to 44 SEAs. District awareness of their state's CCSS 
adoption increased significantly with 86 percent of districts reporting awareness in 2010-11 and 98 
percent in 2011-12 (figure ES-2). 

The largest increases for this reform area were seen in district and school implementation of 
professional development related to the new or revised standards (figures ES-1, ES-2, and ES-3). The 
number of SEAs reporting that they provided, guided, or funded professional development on the CCSS 
increased from 37 to 45, while the percentage of districts in CCSS states that reported providing 
professional development on new or revised state content standards for educators who teach or mentor 
mathematics or reading/English language arts increased from 59 percent to 73 percent. The percentage 
of schools in CCSS states reporting teachers received professional development on new or revised state 
content standards increased from 63 percent to 78 percent. 

In addition, the number of SEAs reporting that they provided instructional materials or curriculum 
assistance for the CCSS increased from 29 to 42, while the percentage of districts reporting distributing 
instructional materials or providing guidance on curricula aligned with new or revised state content 
standards increased from 53 percent to 63 percent, and the percentage of schools reporting use of 
aligned materials increased from 60 to 66 percent. 


9 No data are reported for 2009-10 because the CCSS were not yet available. 
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While the percentage of schools in CCSS states reporting that their teachers received professional 
development targeted to help English learners or students with disabilities master new or revised state 
content standards increased from 62 to 68 percent, the 58 percent of districts reporting providing such 
professional development in 2011-12 did not represent a significant increase over time. 

Figure ES-1. Number of state education agencies (SEAs) that implemented standards and assessments 
reforms: 2010-11 and 2011-12 
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Figure Reads: Forty-three SEAs reported that in 2010-11 they had adopted Common Core State Standards (CCSS) for both 
mathematics and reading/English language arts. Forty-six SEAs did so by 2011-12. 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in 2010-11, the SEA is counted as 
meeting the indicator in 2011-12. 

Notes: Respondents include 50 states and DC. No data are reported for 2009-10 because the CCSS were not yet available. 
Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 State Education Agency Surveys. 
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Figure ES-2. Percentage of districts in Common Core State Standards (CCSS) states that implemented 
reforms related to new or revised state standards: 2010-11 and 2011-12 
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§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 

Figure Reads: Eighty-six percent of districts in Common Core State Standards (CCSS) states reported that in 2010-11 they were 
aware that their state had adopted these standards. In 2011-12, 98 percent of districts in CCSS states were aware that their 
state had adopted these standards. 

Note: No data are reported for 2009-10 because the CCSS were not yet available. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 District Surveys. 
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Figure ES-3. Percentage of schools in Common Core State Standards (CCSS) states that implemented 
new or revised state standards: 2010-11 and 2011-12 
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§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 


Figure Reads: Sixty-three percent of schools in Common Core State Standards (CCSS) states reported that in 2010-11 their 
teachers received professional development on new or revised state content standards. In 2011-12, 78 percent of schools in 
CCSS states reported that their teachers received such professional development. 

Note: No data are reported for 2009-10 because the CCSS were not yet available. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. 
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Data Systems 

Many of the Recovery Act's programs provided incentives to ensure that teachers, schools, 
districts, SEAs, and other stakeholders have information about individual student outcomes, from early 
childhood through higher education and workforce entry, to drive educational improvement. The 
Recovery Act programs emphasized the importance of comprehensive statewide longitudinal data 
systems. Through the assurance areas, the act specified that as a condition for receiving funds, states 
will establish systems that include the 12 core system components described in the 2007 America 
COMPETES Act (Public Law 110-69). 10 Through an infusion of an unprecedented $250 million into the 
Statewide Longitudinal Data Systems (SLDS) grant program, the act also provided funds to build and 
improve such systems. 11 For RTT, each state had to demonstrate the extent to which its statewide 
longitudinal system included all 12 components. The Recovery Act also tunneled an additional $200 
million to the Teacher Incentive Fund (TIF) program. A core element for TIF-funded performance-based 
compensation systems is a data system that can link student achievement data to teacher and principal 
payroll and human resources systems. 

In addition to building comprehensive data systems, the Recovery Act programs encouraged 
states to promote data access and use, and included incentives for districts and schools to use the data. 
States applying for RTT were required to demonstrate how they would make statewide longitudinal data 
accessible to key stakeholders, and how the state would support districts in using data to improve 
instruction. In addition, to receive a share of the $3 billion additional funds for SIG the act made 
available, districts had to commit to implementing specific intervention models in their lowest 
performing schools. Use of student data to inform and differentiate instruction are part of two of the 
four SIG school intervention models, and encouraging evaluation of educators based on student growth 
is part of a third SIG model. Figures ES-4 through ES-6 below summarize state, district, and school 
findings for the study's reform indicators for data systems. 

Implementation and Progress from 2009-10 to 2011-12 

Thirteen SEAs reported operating a longitudinal data system that included all 12 core 
components by 2010-11, unchanged from 2009-10 (data from 2011-12 were not available). While more 
SEAs facilitated access to assessment data and provided professional development or technical 
assistance on their use in 2011-12 than 2009-10, there was no corresponding district trend (figures ES-4 
and ES-5). The number of SEAs reporting that they facilitated access to assessment data and provided 
professional development or technical assistance to support educators' use of assessment data 
increased from 35 to 45 and 32 to 47, respectively, from 2009-10 to 2011-12. However, there was no 
statistically significant increase in the percentage of districts reporting that they provided educators 
with access to assessment data and professional development on the use of assessment data for 
instructional planning. Though, in 2011-12 89 percent and 80 percent of districts reported doing so. For 


10 The 12 components include: use of unique student identifiers; student-level enrollment, demographic, and program 
participation information; exit drop out, transfer, and completion information for P-16 programs; communication with higher 
education data systems; assessing data quality, validity, and reliability; yearly test records; information on students not 
tested; teacher identification systems that allow linking to students; student-level transcript information; college readiness 
test scores; information on students' transition from high school to postsecondary institutions; and other information to 
determine alignment and preparedness for success in postsecondary education. 

11 Prior to the Recovery Act, the SLDS grant program (through three grant competitions) funded 41 states and the District of 
Columbia to design, develop, and implement statewide longitudinal data systems. 
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only one reform at the district level (used longitudinal data to track student achievement gains for 
individual teachers) was there a significant increase from 2009-10 to 2011-12 in the percentage of 
districts implementing (from 60 to 66 percent). 

However, there was a statistically significant increase in the percentage of schools using student 
data to support instruction from 2009-10 to 2011-12 for all reform indicators (figure ES-6). The 
percentage of schools reporting that teachers had online access to student assessment results increased 
from 85 to 92 percent, the percentage of schools reporting that they used student assessment data to 
tailor instruction increased from 85 to 95 percent, and the percentage reporting that they used 
assessment data to identify students for additional support increased from 93 to 98 percent. In addition, 
a higher percentage of schools reported that they used longitudinal data to track student achievement 
gains for individual teachers, increasing from 60 percent in 2009-10 to 71 percent in 2011-12). 



Figure ES-4. Number of state education agencies (SEAs) that implemented data system reforms: 2009- 
10, 2010-11, and 2011-12 
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Figure Reads: Thirty-five SEAs reported that in 2009-10 they facilitated educators' access to assessment data. Forty-three SEAs 
did so in 2010-11, and 45 did so in 2011-12. 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in a previous year, the state is 
counted as meeting the indicator in subsequent years. 

Note: Respondents include 50 states and DC. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 State Education Agency Surveys and U.S. Department of 
Education, State Fiscal Stabilization Fund Initial Annual State Reports (2009-10) and Amended Applications (2010-11). 

States' status on the core components for 2011-12 was not available from the U.S. Department of Education. 
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Figure ES-5. Percentage of districts that supported reforms related to educators' use of student data: 
2009-10, 2010-11, and 2011-12 
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Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < .05). 

§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 

Figure Reads: Ninety-one percent of districts reported that they provided educators with access to assessment data in 2009-10. 
In 2010-11, 88 percent of districts reported that they provided educators with access to assessment data, and 89 percent of 
districts did so in 2011-12. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 District Surveys. 
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Figure ES-6. Percentage of schools that used student data to support instruction: 2009-10, 2010-11, 
and 2011-12 
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A Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < .05). 

+ Percentage for 2011-12 is significantly different from percentage for 2009-10 (p < .05). 

Figure Reads: Ninety-three percent of schools reported that they used student assessment data in 2009-10 to identify students 
for additional support. In 2010-11, 98 percent of schools reported using student assessment data for that purpose, and 98 
percent did so in 2011-12. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. 
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Educator Workforce Development 


The Recovery Act provided substantial incentives for states and districts to work to increase 
teacher effectiveness and the equitable distribution of effective teachers. The act's component 
programs emphasized efforts to develop a more effective educator workforce through the preparation 
of new educators and the adoption of educator evaluation and compensation policies to promote the 
recruitment, retention, and equitable distribution of those educators who were determined to be 
effective. As part of these reforms, the Recovery Act promoted, and under some grant programs 
required, that the effectiveness of teachers and principals be, at least partially, demonstrated through 
growth in their students' achievement. 

The Recovery Act provided incentives for broader implementation of these strategies across the 
country and at the state, district, and school levels. The priorities, reporting requirements, and 
incentives contained in the act's various grant programs encouraged state leadership on educator 
workforce efforts, both in areas where SEAs historically had played a strong role (e.g., teacher licensure 
and certification) and in areas where SEAs had been less involved (e.g., teacher evaluation and pay). In 
particular, both the RTT selection priorities and the SFSF requirements to report on the use of student 
growth as an evaluation criterion and on the distribution of educator performance ratings within 
districts encouraged states to pay more attention to rigorous educator evaluation. The additional 
funding for the TIF and SIG programs through the act also extended incentives for evaluation and 
compensation reform directly to districts and schools and independently from states. TIF required both 
compensation differentiation based on student achievement gains and evaluation based substantially on 
such gains. SIG identified compensation differentiation based on student growth as a permissible 
strategy and required evaluation that included consideration of growth as part of the transformation 
model. Figures ES-7 through ES-9 below summarize state, district, and school findings for the study's 
reform indicators for educator workforce development. 

Implementation and Progress from 2009-10 to 2011-12 

Reforms related to educator preparation were the most common type of state level reform 
implemented in 2011-12 and the number of states supporting these reforms increased from 2009-10 to 
2011-12 (figure ES-7). 12 Six more SEAs reported that they simplified or shortened educator licensure 
processes or authorized non-university preparation programs (increased from 33 to 39 SEAs). 13 Ten 
more SEAs reported that they issued standards or guidelines for teacher preparation programs 
(increased from 28 to 38 SEAs). Fifteen more SEAs reported that they issued standards or guidelines for 
principal preparation programs (increased from 20 to 35 SEAs). Fewer SEAs reported supporting reforms 
related to educator evaluation and compensation but there were still increases in the number of SEAs 
implementing these reforms over time. Eight more SEAs reported differentiating teacher compensation 
based on student achievement gains. Sixteen more SEAs reported that they supported the use of 
student achievement gains for principal evaluation (increased from 6 in 2009-10 to 22 in 2011-12), 
making this the largest increase in implementation found among all state level reform efforts. Thirteen 
more SEAs reported supporting the use of multi-level ratings, multiple observations, and student 
achievement gains for teacher evaluation (an increase from 1 in 2009-10 to 14 in 2011-12). 


Because these indicators measure state activities that require a one-time action and may remain in effect across years, SEAs 
that reported reforms in either of the previous years were considered as continuing the activity in 2011-12. 

13 See Appendix table E.l for the number of SEAs that implemented each of these reforms. 
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At the district level (figure ES-8), the most frequently reported reform in 2011-12 was providing 
school leaders with professional development or flexibility to hire effective teachers; however, providing 
this professional development or flexibility decreased from 69 percent of districts in 2009-10 to 63 
percent in 2011-12 (figure ES-8). 14 Reforms in teacher compensation, teacher and principal evaluation, 
and use of student achievement gains were less commonly reported in 2011-12 (8 to 30 percent). And 
from 2009-10 to 2011-12 there was a decrease in district use of student achievement gains for teacher 
tenure, dismissal, or assignment decisions (from 38 percent to 24 percent). 

Among the educator compensation and evaluation practices that schools were asked about 
(figure ES-9), principal evaluation that included student achievement gains was the most often reported 
reform in 2011-12 (by 49 percent of schools). Thirty-four percent of schools reported that teacher 
tenure, dismissal, or reassignment decisions used student achievement gains in 2011-12, a significant 
increase from the 29 percent that reported doing so in 2009-10. Eighteen percent of the schools 
reported that teacher evaluation practices included multi-level rubrics, multiple observations, and 
student achievement gains in 2011-12. Ten percent of schools reported that teacher compensation was 
differentiated based on student achievement gains in 2011-12. Schools were not asked about reform 
efforts typically under the purview of states and districts, including reforms related to educator 
preparation and flexibility in hiring. 


14 See Appendix table E.5 for the percentage of districts that implemented each of these reforms. 
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Figure ES-7. Number of state education agencies (SEAs) that implemented educator workforce 
development reforms: 2009-10, 2010-11, and 2011-12 
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SEA issued standards or guidelines for 
principal preparation programs 1 


SEA supported differentiating teacher 
compensation based on student 
achievement gains 

SEA supported use of student achievement 
gains for principal evaluation 


SEA supported use of multi-level ratings, 
multiple observations, and student 
achievement gains for teacher evaluation 



Number of SEAs 


Figure Reads: Thirty-three SEAs reported that in 2009-10, they simplified or shortened the educator licensure process or 
authorized non-university educator preparation programs. Thirty-five SEAs did so by 2010-11 and 39 in 2011-12. 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in a previous year, the state is 
counted as meeting the indicator in subsequent years. 

Notes: Respondents include 50 states and DC. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 State Education Agency Surveys. 
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Figure ES-8. Percentage of districts that implemented reforms related to educator workforce 
development: 2009-10, 2010-11, and 2011-12 


District provided school leaders with 
professional development or flexibility 
to hire effective teachers 


District used student achievement 
gains for teacher tenure, dismissal, or 
assignment decisions 


District operated a principal evaluation 
system that included student 
achievement gains 


District operated a teacher evaluation 
system that included multi-level 
rubrics, multiple observations, and 
student achievement gains 

District differentiated teacher 
compensation based on student 
achievement gains 



80 


90 


Percent of Districts 


tPercentage for 2011-12 is significantly different from percentage for 2009-10 (p < .05). 

§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 

Figure Reads: Sixty-nine percent of districts reported that in 2009-10 they provided school leaders with professional 
development or flexibility to hire effective teachers. In 2010-11, 64 percent of districts provided school leaders with 
professional development or flexibility to hire effective teachers, and 63 percent of districts did so in 2011-12. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 District Surveys. 
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Figure ES-9. Percentage of schools that used student achievement gains for educator evaluation, 
compensation, and personnel decisions: 2009-10, 2010-11, and 2011-12 


Principal evaluation practices 
included student achievement 
gains 


Teacher tenure, dismissal, or 
reassignment decisions used 
student achievement gains 


Teacher evaluation practices 
included multi-level rubrics, 
multiple observations, and 
student achievement gains 


Teacher compensation 
differentiated based on 
student achievement gains 



Percent of Schools 


Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < .05). 

+ Percentage for 2011-12 is significantly different from percentage for 2009-10 (p < .05). 

5 Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 

Figure Reads: Forty-five percent of schools used principal evaluation practices in 2009-10 that included student achievement 
gains. In 2010-11, 50 percent of schools used such evaluation practices, and in 2011-12, 49 percent of schools did so. 
Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. 
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Improving Low-Performing Schools 


Almost all of the education programs the Recovery Act funded included a focus on improving 
low-performing schools. These programs offered a multi-pronged approach to improvement that 
included identifying and providing resources to these schools and increasing the quality of their teachers 
and leaders. 

The Recovery Act continued the federal policy of identifying and providing resources to low- 
performing schools, 5 * * * * * * * * * 15 but also brought attention, substantial resources, and intensive reform 
requirements to the persistently lowest-achieving (PLA) schools. These are the state's lowest-achieving 5 
percent of schools and secondary schools with chronically low graduation rates. 16 The act provided an 
additional $3 billion for the SIG program. This large infusion of funds came with new program 
requirements. In general, SEAs are required to (1) identify PLA schools, (2) competitively award SIG 
funds to districts that commit to implementing specific intervention models, and (3) provide technical 
assistance to districts and participating schools. The required intervention models— the transformation, 
restart, closure, and turnaround models— focus on significant changes to a school such as leadership 
and staff changes, closure, or increased student learning time. Districts have the critical role of 
implementing the models and monitoring the progress of each SIG school. The SFSF and RTT programs 
also emphasized the focus on PLA schools, with roles for states to report on PLA schools or support the 
districts that are implementing the school intervention models. 

The Recovery Act programs encouraged the equitable distribution of effective teachers and 
principals in schools. The RTT selection criteria considered state and district plans to ensure that 
students in high-need schools are taught by effective educators. SFSF required SEAs to report on the 
distribution of effective teachers. The act also increased funding for TIF, which enabled an increase in 
the number of grants for districts to develop compensation systems that could entice highly effective 
educators to serve in low-performing schools. 

The Recovery Act also promoted a more favorable environment for public charter schools, 
potentially increasing the pool of these schools available for public school choice. Through the SFSF and 
RTT programs, states were encouraged to remove prohibitions on the existence of these schools and 
limitations to their numbers and to promote accountability for these schools. Figures ES-10 through 
ES-12 below summarize state, district, and school findings for the study's reform indicators for 


5 For example, the act provided $10 billion for Title I, Part A allowing the program to serve more students and improve the 

quality of services provided by implementing evidenced-based strategies to build capacity for improving teaching and learning 

in Title I schools (U.S. Department of Education, 2010c). 

16 Persistently lowest-achieving schools means, as determined by the State: (a) Any Title I school in improvement, corrective 

action, or restructuring that — (i) Is among the lowest-achieving five percent of Title I schools in improvement, corrective 

action, or restructuring or the lowest-achieving five Title I schools in improvement, corrective action, or restructuring in the 

State, whichever number of schools is greater; or (ii) Is a high school that has had a graduation rate that is less than 60 

percent over a number of years; and (b) Any secondary school that is eligible for, but does not receive. Title I funds that — (i) Is 

among the lowest-achieving five percent of secondary schools or the lowest-achieving five secondary schools in the State that 

are eligible for, but do not receive. Title I funds, whichever number of schools is greater; or (ii) Is a high school that has had a 

graduation rate that is less than 60 percent over a number of years. (U.S. Department of Education, November 2010, 

Guidance on FY2010 School Improvement Grants, p. 1. downloaded from: 
http://www2.ed.gov/programs/sif/sigguidancell012010.pdf ). 
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improving low-performing schools. Note that figure ES-11 is limited to districts with low-performing 
schools for 2011-12, 17 and figure ES-12 is limited to low-performing schools. 18 

Implementation and Progress from 2009-10 to 2011-12 

The number of SEAs providing guidance on choosing and implementing school intervention 
models increased during the period from 42 SEAs to include all 51 SEAs by 2011-12. However, reports 
from low-performing schools and their districts suggest that relatively little use was made of key 
elements of the SIG closure, restart, and turnaround models (figures ES-10, ES-11, and ES-12). The 
percentage of districts with low-performing schools reporting targeting low-performing schools for 
closure was 2 percent for 2011-12; the percentage contracting with external organizations to operate 
low-performing schools (expected under the SIG restart model) was 3 percent; and the percentage 
replacing the principal and teachers (expected under the SIG turnaround model) was 5 percent. At the 
school level, the percentage of low-performing schools reporting that a substantial number of teachers 
were replaced was 11 percent and had not increased significantly between 2009-10 and 2011-12, as 
would be expected if more use was being made of the turnaround model. In addition, there was no 
significant change in the percentage of low-performing schools that reported that effective teachers 
were reassigned to the school as part of school restructuring (5 percent in 2011-12). 

The implementation of compensation incentives to improve staffing at low-performing schools 
remained relatively low at the state, district, and school levels during the study period. Between 2009- 
10 and 2011-12, the number of SEAs reporting supporting compensation incentives to improve staffing 
at low-performing schools increased from 8 to 14. Sixteen percent of districts with low-performing 
schools reported in 2011-12 that they provided compensation incentives to improve staffing in low- 
performing schools. Five percent of the low-performing schools reported the use of compensation 
incentives for teachers who move to teach at the school, and there was no increase in this percentage 
between 2009-10 and 2011-12. 

There was an increase in other SEA-level reforms to support improvement in low-performing 
schools between 2009-10 and 2011-12, particularly regarding the expansion of the number of charter 
schools. The number of SEAs that allowed for this expansion increased from 19 to 33 SEAs. The number 
of SEAs that monitored the deployment of effective educators in low-performing schools increased from 
7 to 12 SEAs. 


17 The analysis of districts with low-performing schools was limited to the 2011-12 school year for two reasons. First, districts 
did not report whether they had low-performing schools in 2009-10. Second, the size of the confidence intervals for analyses 
of 2010-11 data raised questions about the accuracy of the estimates. 

18 For this report, low-performing schools include: (1) any ESEA Title l-eligible school designated for improvement, corrective 
action, or restructuring and (2) any high school, regardless of Title I funding or status, with a cohort graduation rate 

(i.e., percent of ninth graders who graduate within 4 or 5 years) less than 60 percent over the last several years. 
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There was also an increase in several of the school-level reforms assessed. The percentage of 
low-performing schools that reported implementing programs to encourage family and community 
involvement increased from 88 to 98 percent, the percentage that reported implementing programs to 
address students' social and emotional needs increased from 81 to 91 percent, and the percent that 
reported implementing programs to orient parents to school improvement models increased from 69 to 
79 percent (figure ES-12). Reforms relating to scheduling and organization were reported by significantly 
greater percentages of low-performing schools in 2011-12 than in 2009-10 (figure ES-13). For example, 
the percentage of low-performing schools that reported modifying the daily schedule to increase 
instructional time for reading/English language arts or mathematics increased from 58 percent to 
78 percent. 



Figure ES-10. Number of state education agencies (SEAs) that implemented reforms to support 
improvement in low-performing schools: 2009-10, 2010-11, and 2011-12 


SEA provided guidance on choosing and 
implementing school intervention models 
defined by ED 1 


State allowed for expansion of the number 
of charter schools 2 


SEA supported using compensation 
incentives to improve staffing at low- 
performing schools 


SEA monitored deployment of effective 
educators in low-performing schools 



Number of SEAs 


Figure Reads: Forty-two SEAs reported that in 2009-10 they provided guidance on choosing and implementing the school 
intervention models defined by ED. Fifty SEAs did so in 2010-11, and all SEAs did so in 2011-12. 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in a previous year, the state is 
counted as meeting the indicator in subsequent years. 

2 In measuring state reform activity for this indicator, if in a previous year a state passed legislation to either increase the 
permissible number of charter schools or remove prohibitions on charter schools, the state is counted as meeting the 
indicator in subsequent years. 

Note: Respondents include 50 states and DC. 

Sources: National Alliance for Public Charter Schools report: Measuring Up to the Model: A Ranking of State Charter School 
Laws (2010 and 2011) and U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education 
Reform: An Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 State Education Agency Surveys. 
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Figure ES-11. Percentage of districts with low-performing schools that implemented reforms to 
support improvement in low-performing schools: 2011-12 


District implemented programs in low-performing schools to 
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District provided technical assistance to low-performing 
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models 
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District provided school leaders in low-performing schools with 
staffing or budgeting flexibility to implement school reform 
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at low-performing schools 
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District contracted with external organization to operate low- 
performing schools 

District targeted low-performing schools for closure 



Percent of Districts with Low-performing Schools 


Figure Reads: Seventy-eight percent of districts with low-performing schools reported that in 2011-12 they implemented 
programs to encourage family and community involvement. 

Notes: Low-performing schools include: (1) any Elementary and Secondary Education Act Title l-eligible school designated for 
improvement, corrective action, or restructuring and (2) any high school, regardless of Title I funding or status, with a cohort 
graduation rate (i.e., percent of ninth graders who graduate within 4 or 5 years) less than 60 percent over the last several years. 
In this figure, the denominator is the estimated number of districts that reported in the they had low-performing schools. 
Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Figure ES-12. Percentage of low-performing schools that supported improvement: 2009-10, 2010- 
11, and 2011-12 
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Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < .05). 

+Percentage for 2011-12 is significantly different from percentage for 2009-10 (p < .05). 

5 Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 


Figure Reads: Eighty-eight percent of low-performing schools reported that in 2009-10 they implemented programs to 
encourage family and community involvement. Ninety-five percent of low-performing schools reported implementing these 
programs in 2010-11, and 98 percent of schools reported implementing these programs in 2011-12. 

Notes: Low-performing schools include schools that were (1) in improvement, corrective action, or restructuring, (2) identified 
as among the lowest-achieving schools; or (3) have had a graduation rate below 60 percent over a number of years. In this 
figure, the denominator is the estimated number of low-performing schools. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. Approved state applications for School 
Improvement Grants for low-performing schools data. Retrieved December 2010 from http://www2.ed.gov/programs/sif/ . 
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Challenges Reported by SEAs, Districts, and Schools in Their Implementation of 
Reforms 

In addition to measuring the extent of reform activity, the study examined challenges that SEAs, 
districts, and schools reported when implementing reforms in 2011-12. The surveys asked states, 
districts, and schools that were implementing any reforms in an assurance area whether they 
encountered particular challenges and whether each challenge was a major or minor one. We present 
for each assurance area, the challenge that was most frequently rated as a major challenge among SEAs, 
districts, and schools that were implementing the reform (table ES-1). 19 For context, the table includes 
the number of SEAs or percentage of districts and schools that rated the challenge (i.e., the number of 
SEAs or percentage of districts and schools engaged in a relevant reform effort). 

• Standards and assessments. At the SEA level, the most frequently reported major challenge in 
2011-12 was a lack of SEA staff or expertise to provide districts with professional development 
or technical assistance on developing interim or formative assessments to measure student 
mastery of the new or revised state content standards (16 of the 32 SEAs rating this challenge 
perceived it as a major challenge). 

At the district and school levels, insufficient funding for supports to help implement the new or 
revised state content standards was the top major challenge. Sixty percent of districts in CCSS 
states identified insufficient funding to purchase new instructional materials aligned with new 
standards as a major challenge in 2011-12. Forty-three percent of schools in CCSS states 
identified insufficient funding to support instructional specialists or coaches to help teachers 
implement new standards as a major challenge in 2011-12. 

• Data systems. While restrictions in rules and regulations on linking of student data to individual 
teachers was the most frequently reported major challenge in 2011-12 by SEAs (14 of the 42 
SEAs rating this challenge perceived it as a major challenge), the top challenge reported by 
districts and schools was delays in transmission of assessment results to schools or teachers. 
Thirty-five percent of districts rated this challenge as a major challenge in 2011-12 as did 21 
percent of schools. 

• Educator workforce development. Difficulty measuring student growth for teachers of subjects 
with no standardized tests was rated as a major challenge by 35 of the 46 SEAs rating this 
challenge in 2011-12. 

Although not asked at the SEA level, large percentages of both districts and schools rated 
insufficient funding to provide performance-based compensation or differential compensation 
as major challenges in 2011-12. Eighty-four percent of the districts rating this challenge 
perceived insufficient funding to provide differential compensation for teachers in high-need 
areas (e.g., science, technology, engineering, and mathematics subjects) as a major challenge. 
Seventy-three percent of the schools rating this challenge perceived insufficient funding to 
provide performance-based compensation to all eligible teachers as a major challenge. 


19 


Each challenge could be rated as not a challenge, a minor challenge, a major challenge, or not applicable. 



• Improving low-performing schools. Fifteen of the 49 SEAs rating this challenge reported 
restrictions in rules and regulations regarding the extent of autonomy that LEAs and schools can 
be granted in terms of staffing or budgets as a major challenge in 2011-12. Insufficient funding 
to implement whole-school or turn around intervention models was the challenge most 
frequently reported as major by districts with low-performing schools 2011-12 (by 65 percent of 
districts rating the challenge). At the school level, restrictions in rules and regulations on 
replacing less effective teachers was the challenge most frequently reported as major (by 49 
percent of low-performing schools rating the challenge). 

The study also investigated whether the percentage of SEAs rating challenges as major differed for 
states that received an RTT grant compared with those that did not. 20 

• With the exception of several challenges related to the improvement of low-performing schools 
and one challenge related to educator workforce development, in 2011-12 a smaller percentage 
of SEAs in RTT states rated challenges as major than did SEAs in other states. 21 


20 At the district level, we examined reports of major challenges by district size and poverty status. We examined school reports 
of major challenges by performance status. These analyses did not identify clear patterns and are not part of this report. 

21 See chapter 6 for detailed findings. 
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Table ES-1. Top major challenge when implementing reforms, as reported by state education 
agencies (SEAs), districts, and schools, by assurance area: 2011-12 


Assurance area and level 

Challenge most frequently reported as major 

Percent that 
reported 
challenge as a 
major challenge 

Standards and 
assessments 1 

SEA 

Lack of SEA staff or expertise to provide districts with professional 
development or technical assistance on developing interim or 
formative assessments to measure student mastery of the new or 
revised state content standards (16 of the 32 SEAs rated the 
challenge and perceived it as a major challenge) 

50 

District 

Insufficient funding to purchase new instructional materials 
aligned with new standards (95 percent of districts rated the 
challenge) 

60 

School 

Insufficient funding to support instructional specialists or coaches 
to help teachers implement new standards (84 percent of schools 
rated the challenge) 

43 

Data systems 

SEA 

Restrictions in rules and regulations on linking of student data to 
individual teachers (14 of the 42 SEAs rated the challenge and 
perceived it as a major challenge) 

33 

District 

Delays in transmission of assessment results to schools or teachers 
(94 percent of districts rated the challenge) 

35 

School 

Delays in transmission of assessment results to school or teachers 
(93 percent of schools rated the challenge) 

21 

Educator 

workforce 

development 

SEA 

Difficulty in measuring student growth for teachers in non-tested 
subjects (35 of the 46 SEAs rated the challenge and perceived it as 
a major challenge) 

76 

District 

Insufficient funding to provide differential compensation for 
teachers in high-need areas (e.g., low- performing schools, science, 
technology, engineering, and mathematics subjects) (53 percent of 
districts rated the challenge) 

84 

School 

Insufficient funding to provide performance-based compensation 
to all eligible teachers (49 percent of schools rated the challenge) 

73 

Improving low- 

performing 

schools 

SEA 

Restrictions in rules and regulations regarding the extent of 
autonomy that LEAs and schools can be granted in terms of 
staffing or budgets (15 of 49 SEAs rated the challenge and 
perceived it as a major challenge) 

31 

District 2 

Insufficient funding to implement whole-school or turn around 
intervention models (65 percent of districts rated the challenge) 

65 

School 3 

Restrictions in rules and regulations on replacing less effective 
teachers (76 percent of schools rated the challenge) 

49 


1 Percentages limited to SEAs, districts, and schools in states that adopted the Common Core State Standards (CCSS). 

2 Percentages limited to district with low-performing schools. 

3 Percentages limited to low-performing schools. 

Table Reads: In 2011-12, 50 percent of SEAs in CCSS states rating this challenge perceived lack of SEA staff or expertise to 
provide districts with professional development or technical assistance on developing interim or formative assessments to 
measure student mastery of the new or revised state content standards as a major challenge to implementing new or revised 
state standards and aligned assessments. Thirty-two SEAs rated this challenge. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey, District Survey, and School Survey. 
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Chapter 1: Introduction 


The American Recovery and Reinvestment Act (ARRA or the Recovery Act) of 2009 provided an 
unprecedented level of funding for K-12 education. The program created a "historic opportunity to save 
hundreds of thousands of jobs, support states and school districts, and advance reforms and 
improvements that will create long-lasting results for our students and our nation." 22 As a way of 
promoting educational improvement, the Recovery Act required recipients of ARRA funds to commit to 
reforms in four key policy areas: standards and assessments, data systems, teacher effectiveness, and 
low-performing schools. 

This is the final report of a multi-year U.S. Department of Education (ED) evaluation, Charting 
the Progress of Education Reform: An Evaluation of the Recovery Act's Role. ED sought to track the 
adoption of policies and implementation of practices associated with the education reforms promoted 
by the Recovery Act at the state, district, and school levels. Other reports have examined the extent to 
which the Recovery Act saved and/or created jobs, 23 the distribution of Recovery Act education grants 
to states and school districts, 24 and whether, and how, state education agencies (SEAs) implemented the 
reforms that the Recovery Act emphasized 1 year after the act was passed. 25 

This report focuses on the implementation of reforms promoted by the Recovery Act 2 full 
school years after all Recovery Act funds were awarded (i.e., the 2011-12 school year). It describes the 
status of reform implementation at this time for states, districts, and schools nationwide and 
investigates whether implementation varied by key characteristics such as whether the state received a 
Race to the Top (RTT) award funded by the Recovery Act, whether the district was a high-poverty 
district, and whether the school was a low-performing school. The report provides information on major 
challenges to reform implementation in 2011-12, based on whether states, districts, and schools rated 
potential challenges (e.g., lack of staff or expertise, lack of funding) as a major challenge. Finally, since 
the study collected information about the status of activities in 2009-10, the report includes a discussion 
of the progress of reform implementation from the time Recovery Act funds were just being distributed. 


22 The American Recovery and Reinvestment Act of 2009: Saving and Creating Jobs and Reforming Education (March 2009). 
http://www2.ed.gov/policv/gen/leg/recovery/implementation.html 

23 See for example, U.S. Department of Education, (n.d.). ED Recovery Act Jobs Report (September 30, 2010). Retrieved from 
http://www2.ed.gov/policv/gen/leg/recovery/spending/impact5.html . (This site includes access to all quarterly ED Recovery 
Act jobs reports on Recovery Act spending by state through September 30, 2010.) Congressional Business Office. (February 
2012). Estimated Impact of the American Recovery and Reinvestment Act on Employment and Economic Output from October 
2011 through December 2011. Retrieved from http://www.cbo.gov/sites/default/files/cbofiles/attachments/02-22-ARRA.pdf . 
Information about jobs saved or created for each state under the SFSF can be found in each state's annual performance 
report, see: http://www2.ed.gov/programs/statestabilization/annual-reports.html . 

24 Garrison-Mogren, R., and Gutmann, B. (2012). State and District Receipt of Recovery Act Funds— A Report from Charting the 
Progress of Education Reform: An Evaluation of the Recovery Act's Role (NCEE 2012-4057). Washington, DC: National Center 
for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Available at 
http://ies.ed.gov/ncee . 

25 Webber, A., Troppe, P., Milanowski, A., Gutmann, B., Reisner, E., and Goertz, M. (2014). State Implementation of Reforms 
Promoted Under the Recovery Act— A Report From Charting the Progress of Education Reform: An Evaluation of the Recovery 
Act's Role (NCEE 2014-4011). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of 
Education Sciences, U.S. Department of Education. Available at http://ies.ed.gov/ncee . 
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The Recovery Act 


The Recovery Act allocated $70.6 billion in funding for K-12 education, of which $68.8 billion 
was awarded to states through a combination of newly created and existing grant programs. Through 
the new State Fiscal Stabilization Fund (SFSF), ED awarded $39 billion in Education State Grants. ED 
distributed these grants to all states by formula, and they were primarily intended to help fill state 
budget shortfalls and to save and create jobs, including those of teachers and school administrators. 
New discretionary grant programs included Race to the Top (RTT) and the Investing in Innovation Fund 
(i3), which awarded $3.9 billion and $646 million, respectively, and which were intended primarily to 
support education reforms. 

In addition, 10 existing programs, both formula funded and competitive, received a $24.9 billion 
infusion of funds to support activities authorized under general program authority. All states received 
formula grant funds for Individuals with Disabilities Education Act, Part B State Grants and Preschool 
Grants; Title I Part A; School Improvement Grants (SIG); Education for Homeless Children and Youth; and 
State Educational Technology grants ($24.2 billion). ED also awarded additional funds through Impact 
Aid formula grants and through competitive Impact Aid; Statewide Longitudinal Data Systems (SLDS), 
and Teacher Incentive Fund (TIF) grants ($544 million). 26 Most of the additional funds were not explicitly 
focused on promoting reform. The exceptions are the TIF and SIG programs. Even before the Recovery 
Act, the activities authorized for these programs were reform oriented. Authorized activities for 
competitive TIF grants include creating sustainable performance-based compensation systems. The $3 
billion in additional funding for the SIG program was allocated as formula grants to SEAs. SEAs then 
make competitive sub-grants to districts that demonstrate the greatest need for the funds and the 
strongest commitment to use the funds to raise substantially the achievement of students in their 
lowest-performing schools. 

ED allocated most of the Recovery Act formula grant funds by September 2009 and the 
competitive grant awards by September 2010. As described in an earlier report from this evaluation, 
individual states received between $1,063 to $3,632 per pupil from Recovery Act grants for K-12 
education. 27 Much of the variation in funding across states reflects the receipt of competitive grant 
funds, most notably RTT. For most K-12 programs funded by the Recovery Act, states were required to 
subgrant to local education agencies (LEAs) and were encouraged to make these subawards quickly. 
Most LEA subgrants were awarded by the end of 2010. 28 However, for some programs— specifically RTT 
and SIG— it took longer for funds to begin flowing to LEAs and much longer for all funds to be 
distributed. 


For information about the authorized activities for each of these programs, see table 1 of Garrison-Mogren and Gutmann, 
2012 . 

27 See Garrison-Mogren and Gutmann, 2012. 

28 See p. 1 of Garrison-Mogren and Gutmann, 2012, for more information about the specific grant awards to individual states 
and the timing of funds allocated to states and districts. 
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The Four Assurance Areas 


In return for Recovery Act grants, states were required to commit to advancing four specific 
education reform priority areas, or "assurance areas": 

• "Making progress toward rigorous college- and career-ready standards and high-quality 
assessments that are valid and reliable for all students, including English language learners and 
students with disabilities, 

• Establishing pre-K to college and career data systems that track progress and foster continuous 
improvement, 

• Making improvements in teacher effectiveness and the equitable distribution of qualified 
teachers for all students, particularly students who are most in need and 

• Providing intensive support and effective interventions for the lowest-performing schools." 29 

The four assurance areas were intended by the act's drafters to constitute an integrated, 
comprehensive vision of educational improvement that would be capable of raising the academic 
performance of all public school students. The vision embodied in the assurance areas begins with high 
expectations and accountability for student achievement (i.e., shared, rigorous standards and 
appropriate assessments). To support this vision, the Recovery Act's programs encouraged the 
development of data tools that can provide information to both help educators identify needs for 
improvement and provide feedback on the impact of educational changes on student learning. The act 
also recognizes the importance of effective educators by emphasizing improvements in their 
preparation, evaluation, and compensation and on achieving a more equitable distribution of effective 
teachers across schools within LEAs. Finally, to upgrade persistently low-performing schools, the act's 
programs provide incentives and tools for intervening in and improving these schools. 

By linking a commitment to the four assurance areas with receipt of funding, the Recovery Act 
signaled federal priorities; provided states, districts, and schools with incentives to initiate or intensify 
reforms in each of these areas; and encouraged states to pursue a combination of mutually supporting 
reform strategies. The type and strength of incentives embedded in the Recovery Act varied by grant 
program, however. For example, to receive SFSF funds, governors had only to agree to advance the four 
assurance areas with their funding applications. In contrast, the RTT grant competition gave substantial 
weight to states' enactment of specific policies in these assurance areas, such as the adoption of the 
Common Core State Standards (CCSS), participation in one of the multi-state assessment consortia, 
implementation of a statewide longitudinal data system, and the development of performance-based 
teacher evaluation systems. Applicants were also judged on their plans to support the implementation 
of these education reforms that would be funded, in part, by RTT grants. And some programs, like the 
Title I SIG program, were more prescriptive, requiring schools targeted for SIG support to implement 
one of four school improvement models. 


29 The American Recovery and Reinvestment Act of 2009: Saving and Creating Jobs and Reforming Education (March 2009). 
http://www2.ed.gov/policy/gen/leg/recovery/implementation.html 
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The Role of States, Districts, and Schools 

Though states (and their SEAs) are the primary focus of the four assurance areas and the 
primary conduits for the act's financial assistance, understanding the progress of the reforms promoted 
by the act also requires examining reform activity at the district and school levels. This is important 
because (1) states differ in the degree to which state policy determines district and school action, (2) 
state policy regulates or enables activities or programs for which districts or schools are responsible, and 
(3) a few of the act's component programs bypassed the state level to provide funds directly to districts 
promising to undertake specific reforms that were related to the assurance areas. 

Given the decentralized nature of the U.S. education system and the variation in the degree of 
local control across the states, districts have especially important roles in implementing the Recovery 
Act's vision. For example, though the expectation for state leadership on student standards was clear, 
districts have a role in implementing standards. Some states do not require districts to adopt these 
standards or state-developed curricular materials aligned with them. States vary widely in their role in 
providing professional development to educators on topics such as implementing new standards (see, 
for example, Goertz, 2005). States were also expected to make progress toward developing 
comprehensive data systems, but districts use the data to evaluate instructional programs and 
educators. Districts also train educators in data use and provide access to state data though district 
information systems. Responsibility for the quality of the educator workforce is shared. The RTT 
component of the act provided states with incentives to remove barriers to evaluating educators based 
on the achievement of their students, and resources support more rigorous evaluation systems. But 
while states may put regulations into effect or provide assistance in support of new educator evaluation 
or compensation systems, districts as employers evaluate and pay educators. A few Recovery Act 
programs such as i3 and TIF provided money directly to local entities, bypassing the state and providing 
direct incentives for district action. Charting the progress of the Recovery Act's vision thus requires 
attention to district activities related to the four assurance areas. 

While the Recovery Act provided incentives to change state and district policies, it is at the 
school level where policies and programs affect students. Moreover, districts, like states, may adopt 
different strategies for the implementation of the reforms promoted by the act. For example, some 
districts create and mandate the use of a system-wide curriculum, while others delegate the selection of 
curriculum and instructional materials to their schools. Responsibility for the level and content of 
professional development is often shared between schools and districts. Some districts have a 
centralized process for hiring and placing teachers, while other districts leave these decisions in the 
hands of school leaders. Districts also may implement educator compensation or evaluation reforms in 
subsets of schools. Some districts mandate the use of particular school-turnaround models; others leave 
the choice to the school. Some districts have a high proportion of low-performing schools, while others 
have only a small proportion. One of the act's components, SIG, required states to direct most of their 
SIG funds to districts and schools in return for promises of reform at the school level, including the 
adoption of prescribed school-turnaround models. Understanding the reach of the reforms promoted by 
the Recovery Act thus requires an examination of their use at the school level. 
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The Recovery Act and Ongoing Education Reform 


As described in succeeding chapters of this report, the Recovery Act was introduced into an 
ongoing stream of federal, state, and local reform. Many of the reforms promoted within each of the 
four assurance areas were built on prior reform activity or were already being implemented in some 
form. Standards have been a foundational element of state and federal policy since the 1994 
reauthorization of the Elementary and Secondary Education Act (ESEA), and all 50 states had academic 
content standards prior to the Recovery Act. Regarding data systems, the federal government was 
already encouraging states to build student data systems through the Statewide Longitudinal Data 
Systems (SLDS) grants. In the area of educator effectiveness, Florida, Minnesota, North Carolina, and 
Texas had introduced efforts to promote educator effectiveness through compensation reform. In 
addition, the first two rounds of the TIF grants encouraged states and districts to experiment with 
performance-based compensation. Federal concern with low-performing schools has been long 
standing, including both requiring states to identify these schools and providing resources for improving 
them. In parallel, states, districts, and foundations have experimented with a wide variety of 
interventions aimed at improving these schools' performance. 

The Recovery Act program initiatives were intended to build on the successes and address the 
limitations of policies that were in place at the time. For example, states' expectations for students 
differed considerably (National Center for Education Statistics, 2011), and there was concern that state 
standards did not necessarily represent what students need to know and be able to do to succeed in 
college and today's workplace. These shortcomings led to federal incentives for states to adopt the CCSS 
and corresponding assessments. Recovery Act program initiatives in the area of educator effectiveness 
sought to encourage nationwide adoption of the reform efforts on which some states and districts had 
been working and expand the efforts through the TIF program. Funding for the SLDS program was 
increased through the Recovery Act. Sizable increases to SIG funding through the Recovery Act led to 
new program requirements for states to pass along funds competitively to districts and schools willing 
to implement one of four school intervention models. These models both built on prior strategies and 
attempted to promote coherence. 

Because of the differing roles of states, districts, and schools in education reform; variation in 
previous reform activities by these levels; and the different design and foci of the programs making up 
the Recovery Act, this study was designed to capture implementation of education reforms within each 
assurance area at all three levels. The next section of this chapter provides an overview of the key 
research questions and the study's methodology. 


Study Questions and Methods 


This study was designed to capture implementation of the reform agenda promoted by the 
Recovery Act at the state, district, and school levels for each of three school years: 2009-10, 2010-11, 
and 2011-12. This report focuses on reform implementation as of 2011-12, the second full school year 
after all Recovery Act funds were awarded. The implementation of reforms, such as the CCSS, new 
assessments, and performance-based teacher evaluation systems, unfolded over multiple years. For 
example, some states were engaged in the planning stage of reforms during 2010-11 and began 
implementation in later years. Because the implementation of certain reforms began before passage of 
the Recovery Act, state, district, and school activity during the 2009-10 school year provided a baseline 
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for the study, capturing the reforms already in place when Recovery Act funding was just beginning to 
be distributed. 

Study Questions 

Specifically, this report addresses the following questions: 

• To what extent did SEAs, districts, and schools report implementing key reform strategies 
promoted by the Recovery Act in the 2011-12 school year? 

• How much of the 2011-12 school year implementation reflects progress since the Recovery Act? 

• Did the extent of reform in the 2011-12 school year vary by relevant state, district, and school 
characteristics? 

• What were the greatest reform implementation challenges for SEAs, districts, and schools in the 
2011-12 school year? 

The report does not assess the relationship between the amount of Recovery Act funds received 
or the receipt of funds from specific grant programs and the implementation of reforms. All states 
committed to the four core reforms or assurance areas as a condition for receiving funds, and more than 
90 percent of the Recovery Act funding was awarded by formula to all states. Furthermore, much of the 
variation in per pupil Recovery Act funding by state appears to be driven by receipt of competitive grant 
funds (Garrison-Mogren and Gutmann, 2012, p. 20). Most of the states with the highest funding per 
pupil are those that won an RTT grant in the first two rounds of this competition. As a result, examining 
reform implementation by state per-pupil Recovery Act funding would primarily pick up differences by 
state RTT status, which this report already examines. 

At the district level, it is difficult to accurately account for all funding (or benefits) received by 
districts from all Recovery Act programs. 30 Additionally, some reforms depend on, or are facilitated by, 
state actions (and likely funded by dollars retained at the state level). An analysis of district funding and 
reform implementation would miss this effect. Finally, the Recovery Act was introduced into an ongoing 
stream of federal, state, and local reform making it difficult to isolate the effect of the Recovery Act on 
state, district, or school adoption of particular reforms. Rather, the report provides a snapshot of the 
status of reforms at particular points in time. The study did not collect information about the reasons 
why states, districts, and schools adopted or implemented these policies and practices. In addition, the 
report does not address whether state, district, or school implementation of more reforms constitutes 
an integrated and comprehensive reform approach. 


] For example, the subgrant data in Recovery.gov do not systematically include subgrants of less than $25,000. For the i3 and 
TIF programs, the grantee could be a non-profit or a consortium of districts, making it difficult to assign award amounts to 
individual districts. 
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Data Sources 


The findings in this report draw primarily from surveys administered to all 50 SEAs and the 
District of Columbia (DC) and nationally representative samples of school districts and schools during 
spring 2011 and spring 2012. For the SEA survey, respondents were the chief state school officer or 
other state agency officials designated by the chief as most knowledgeable about the topics in the 
survey (e.g., associate or deputy superintendent, director of curriculum and instruction, director of 
assessment and accountability). For the district survey, respondents were district staff most 
knowledgeable about the topics in the survey. Principals responded to the school survey. 

We developed the SEA survey to describe state adoption of specific education policies and 
programs (including specific elements of states' evaluation, compensation, and data systems), the state 
role in supporting implementation of these reforms, and challenges in implementing the reforms in each 
of the four assurance areas. The district survey focused on district adoption of specific education policies 
and use of strategies to support and promote reform policies at the school level and asked about 
challenges implementing strategies. The school survey focused on whether specific practices or 
strategies associated with implementing state or district policies or programs related to the assurance 
areas were being used in schools, where reform is most likely to affect students. The survey also asked 
school leaders about the challenges they encountered when implementing specific practices. 

The survey items were designed to capture the key reform practices or strategies ED identified 
in its grant notices, regulations, and guidance for the Recovery Act programs. We drew on the specific 
strategies and activities described in SFSF assurance indicators and descriptors, the RTT selection 
criteria, and guidance for the Title l-ARRA and SIG programs. 31 

While the state, district, and school surveys covered parallel topics, individual survey items were 
tailored to focus on activities most relevant to a particular level. For example, in the case of standards 
and assessments, the SEA survey asked about state adoption of the CCSS and other new or revised 
content standards, then focused on specific state activities that supported the implementation of 
content standards, including professional development, instructional materials, and assistance to LEAs in 
curriculum mapping. The district survey, in turn, asked about the distribution of instructional materials 
to schools and whether the district made available or provided professional development on the new or 
revised state content standards to educators. The school survey asked what practices the school used to 
implement new or revised state content standards such as educators' receipt of professional 
development, use of curriculum frameworks, or use of curriculum aligned with the new or revised state 
content standards. For additional details about survey development, see appendix A. 

The spring 2011 surveys asked about SEA, district, and school activities in 2009-10 and 2010-11. 
The spring 2012 surveys asked about SEA, district, and school activities in 2011-12. 


31 See: U.S. Department of Education. (2010). Overview Information; Race to the Top Fund; Notice Inviting Applications for New 
Awards for Fiscal Year (FY) 2010. Federal Register, 75(71). Available from http://www.gpo.gov/fdsvs/pkg/FR-2009-ll- 
18/pdf/E9-27427.pdf . U.S. Department of Education. (2009, November). State Fiscal Stabilization Fund Program; Final Rule. 
Federal Register, 74(217): 58436-58525. Available from http://www2.ed.gov/legislation/FedRegister/finrule/2009- 
4/111209a.pdf. U.S. Department of Education. (2010). School Improvement Grants; American Recovery and Reinvestment Act 
of 2009 (ARRA); Title I of the Elementary and Secondary Education Act of 1965, as Amended (ESEA). Federal Register, 

75(208): 66363-66371. Available from http://www.gpo.gov/fdsys/pkg/FR-2010-10-28/pdf/2010-27313.pdf 
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Extant data provided information about state-level reform activity in two areas: removing or 
reducing limits on charter schools and characteristics of SLDS. The National Alliance of Public Charter 
Schools publishes an annual review of state charter school laws. We also used annual performance data 
that states reported to ED for the SFSF program to provide information about state education data 
systems. 

District and School Sample Design 

SEAs in the 50 states and the District of Columbia were surveyed. For the district and school 
levels, the study selected nationally representative samples of school districts and schools that were in 
operation in the 2010-11 school year. The district sample of 1,700 school districts was selected using an 
enrollment-driven design, where districts with more students enrolled were more likely to be selected 
than they would have otherwise. We also oversampled high-poverty districts to support subgroup 
analyses for this population of interest. Within the high-poverty stratum and its complement stratum, 
there were geographic strata to ensure that each RTT state had at least one district represented in the 
sample and to balance districts in other states broadly across the U.S. Within each stratum, we sorted 
districts into an ordered list based on their charter status, TIF status, urban-centric locale code, and 
enrollment and sampled systematically using this sorted list. Districts with one school (e.g., these can be 
charter schools that also are a district) were undersampled to minimize the numbers of these districts in 
the final sample, while still representing them in the nationally representative sample. 

Within the sampled districts, we drew a sample of 3,800 schools, selecting at least two sampled 
schools in each district. 32 The school sample used an enrollment-driven design to give larger schools a 
higher chance of selection than they would have otherwise. We oversampled persistently lowest- 
achieving (PLA) schools to support subgroup analyses of low-performing schools. The school sample was 
selected using a statistical method called "balanced sampling," which controlled the number of schools 
sampled for each sampled district and balanced across school grade span (elementary, middle, high, 
other) and school performance (PLA, schools in need of improvement non-PLA, and other schools). 33 See 
appendix A for more details on the district and school sample designs. 

Data Collection 

The surveys were fielded in spring 2011 and spring 2012, using the same sample of districts and 
schools for both administrations. All SEAs responded to both surveys. For the 2011 data collection, 88 
percent of districts (unweighted) and 78 percent of schools responded to the web surveys. The response 
rates increased to 91 percent for districts but decreased to 72 percent for schools in the 2012 data 
collection. District and school estimates and standard errors in this report were based on replicate 
weights that take into account the complex sample design and survey nonresponse adjustments. See 
appendix A for more information on the weighting approach. 


Unless the sampled district had only one school, in which case we selected that school with certainty into the sample. 

33 While enrollment-based probabilities (i.e., probability of selection proportionate to size, or PPS) were used to select the 
district and school samples, the report's research questions ask about the number and percentage of districts and schools 
implementing reforms. Thus, unit-based weights (i.e., those that estimate total numbers of districts and schools) were 
developed and used for the analyses in the report. These provide unbiased estimates, but do have higher variability (i.e., more 
sampling variance, which in turn leads to higher standard errors) than weights based on enrollment. Note that at the school 
level, school enrollment sizes are less variable than the district enrollment sizes, so the design effects from having unit-based 
estimates with a PPS sample are smaller for schools than for districts. See appendix A for more information. 
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Indicators of Reform 


To address the study questions, we developed indicators of reform implementation at the state, 
district, and school levels. The indicators reflect ED's priorities and key reform strategies within each of 
the four assurance areas. The indicators were designed to provide a high-level snapshot of whether 
SEAs, districts, or schools had a particular policy in place, provided support, or carried out a particular 
activity. The indicators do not describe the quality or intensity of reform implementation. As discussed 
below, for most indicators, an SEA, district, or school was said to have met an indicator if it reported 
implementing any one of a particular set of related strategies. For a smaller set of indicators, an SEA or 
district met the indicator only if it met more than one requirement. At the state level, we created 18 
indicators of reform implementation using SEA survey items and extant data. At the district level, we 
created 21 indicators of reform implementation, and at the school level, we created 22 indicators of 
reform implementation from survey data. See appendix B for the components, decision rules, and 
specific Recovery Act requirements embodied in each indicator. Subsequent chapters on each assurance 
area provide context and rationale for each indicator. 

Because of the variety of potential SEA and district responses to Recovery Act reform 
requirements and because assurance areas could be met by using different approaches, the indicators 
often captured several ways in which a state, district, or school might implement a reform. For example, 
SEAs could use multiple strategies in their role to improve standards and assessment, from providing 
professional development directly to supporting LEA professional development through guidance and 
technical assistance. Similarly, districts could use multiple strategies to support or promote the 
implementation of new standards and assessments in schools, from distributing instructional materials 
to providing criteria for schools to use when selecting new aligned curriculum. Where appropriate, the 
indicators include multiple strategies and do not assume that one approach is preferable to another. 
Some Recovery Act programs, however, have more prescriptive requirements. In these cases, states and 
districts had to take specified actions, such as adoption of the CCSS or the inclusion of student growth 
measures in educator evaluation systems, to meet an indicator. 

Finally, some reform activities measured in the SEA indicators have the potential to continue 
over multiple years once they are instituted. For example, when a state issues standards or guidelines 
for teacher preparation programs, these are likely to remain in effect until new or revised standards or 
guidelines are issued. So, this one time action of issuing standards can represent a support for a reform 
effort that is ongoing or continued in future years. In measuring SEA reform activity for selected 
indicators, if a state reported activity in a previous year, the SEA is counted as meeting the indicator in 
subsequent years. See appendix A for additional detail about which SEA activities were considered 
ongoing over multiple years. 

Analysis Methods 

The report presents the prevalence and progress (between 2009-10 and 2011-12) of reform 
implementation at the state, district, and school levels for each assurance area. In addition, we 
examined the status of reform implementation by selected SEA, district, and school characteristics. We 
also report on the number or percentage of SEAs, districts, and schools that rated various potential 
challenges as "major challenges" to implementing reforms in 2011-12. As a summary for each assurance 
area, we describe similarities and differences among the state, district, and school levels in 
implementation progress. For all but the data systems assurance area, we also describe similarities and 
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differences among the three levels on the challenges perceived by those implementing reforms. 34 These 
descriptive analyses are discussed below. 

Measuring status and progress of reform implementation 

For each assurance area, we examined the extent of reform activity at the three levels (SEAs 
districts, and schools) in two ways. We first describe the prevalence of reform by examining the number 
(for SEAs) or percentage (for districts and schools) that met each indicator in 2011-12. This analysis 
provides the basic snapshot of SEA, district, and school status on the Recovery Act reform agenda 2 
school years after all funds were awarded to states (i.e., 2011-12). Second, we compare the number or 
percentage of SEAs, districts, and schools that reported they had a reform already in place in 2009-10 
with the number and percentage that reported they had the reform in place by 2010-11 and 2011-12 to 
get a measure of new activity or progress over time. 35 That is, we discuss the change in the total number 
of SEAs or percentage of districts and schools implementing reforms from one year to the next. Note 
that change over time in the number of SEAs or the percentage of districts and schools represents a net 
change. 36 The surveys did not ask about reforms in place prior to 2009-10, or how long a particular 
reform had been in place. 

Measuring status of reform activity by selected characteristics 

In addition to examining the extent of reform activity across all SEAs, districts, and schools, we 
compared the extent of reform activity in 2011-12 by several SEA, district, and school characteristics to 
determine if certain types of entities were more likely to report implementing reforms than others. 

At the state level, we examined reform implementation by whether the state received a first or 
second round RTT award. Because states received RTT awards based, in part, on their actual and 
planned implementation of reforms similar to those included in the Recovery Act, these states may be 
expected to be farther ahead on these reforms, compared with other states, by 2011-12. 

At the district level, we examined reform implementation by district poverty status and district 
size. 37 High-poverty districts have been a long-standing focus of federal educational policy. These 
districts have had high levels of prior exposure to and participation in federal education programs in 
which funding is directly or indirectly linked to incidence of poverty. These additional resources, plus 
experience complying with related federal requirements, may have built the reform capacity of high- 
poverty districts and disposed them to be especially responsive to federal reform initiatives. In addition, 
since poverty is inversely related to student achievement, stakeholders in higher poverty districts may 


34 A comparison of challenges perceived by SEAs, districts, and schools was not conducted for the data systems assurance area 
because there are no directly comparable challenges common to all three levels. 

35 The analysis of progress over time for the standards and assessments assurance area was limited to the 2010-11 and 2011-12 
school years since the Common Core State Standards were not in place in 2009-10. 

36 It is possible that an individual SEA, district, or school could have discontinued a reform strategy even though the total 
number of SEAs or percentage of districts and schools increased over time. In addition, since the district and school analyses 
were not limited to entities that responded to both surveys, changes in survey respondents from one year to the next can also 
influence the net change observed. 

37 High-poverty districts are defined as those with a child poverty rate above 21.66 percent. District size is based on the student 
enrollment in schools in the district. Large districts have at least 50,000 students. Medium districts have fewer than 50,000, 
but at least 15,000 students. Small districts have fewer than 15,000 students. 
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also perceive a greater need for reform. We therefore compared reform implementation in high-poverty 
districts with reform implementation in other districts to determine whether high-poverty districts were 
more likely to have made progress in implementing Recovery Act reforms. This contrast is also 
motivated by the Recovery Act's focus on equity. 

Because large districts are likely to have developed greater district-wide administrative capacity 
to support educational operations and educational change than districts with smaller enrollments, large 
districts are likely to have greater capacity to implement Recovery Act reforms. For example, prior 
research suggests that larger districts invest more resources in specialized staff and systems in the areas 
of assessment, curriculum support, human resources, and technology. 38 Also, the National Assessment 
of Title I found that very large districts had more staff available to provide assistance to low-performing 
schools than smaller districts. 39 As a result, these districts may possess the specialist expertise and 
infrastructure to implement the Recovery Act's provisions regarding low-performing schools more fully 
than is likely in districts with smaller enrollments. This additional management capacity may also create 
economies of scale in leveraging per-pupil funding allocations toward reform implementation. While 
larger districts are more likely to receive Title I funds, size is not always correlated with poverty (which 
drives Title I funding allocations), and Hannaway and Kimball (1998) found that size was related to 
reform activity after controlling for district poverty. Thus, we examined whether large districts achieved 
higher levels of reform implementation than smaller districts during the period covered by this study. 

For three of the four assurance areas, we also examined differences in reform implementation 
between schools identified as low performing 40 and schools not so identified. We include this 
comparison because of the emphasis throughout the act's education provisions on improving the 
performance of students in low-performing schools. We include these comparative analyses for the 
assurance areas on standards and assessments, data systems and use, and educator workforce 
development. For the assurance on improving low-performing schools, we present findings on reform 
implementation for low-performing schools only. 

Reported challenges in implementing reforms 

We examined challenges that SEAs, districts, and schools reported in implementing reforms in 
2011-12. 41 The surveys asked states, districts, and schools that were implementing any reforms in an 
assurance area (e.g., standards and assessment) whether they encountered a particular challenge and 
whether the challenge was a major or minor one. For each assurance area, we present the number of 
implementing states and the percentage of implementing districts and schools that considered any issue 


See analyses of differences in education capacity among districts based on enrollment size: U.S. Department of Education, 
Office of Planning, Evaluation, and Policy Development, Policy and Program Studies Service, Title I Accountability Systems 
and School Improvement from 2001 to 2004, Washington, D.C., 2006. 

39 Institute of Education Sciences (2007). National Assessment of Title I, Final Report: Volume I: Implementation. Washington, 
DC: U.S. Department of Education, p.74 

40 Low-performing schools include schools that (1) were in improvement, corrective action, or restructuring under Title I, ESEA; 
(2) were identified as among the lowest-achieving schools; or (3) had a graduation rate below 60 percent over a number of 
years. 

41 We also examined SEA reports of major challenges by state RTT status. Chapter 6 presents these findings. At the district level, 
we examined reports of major challenges by district size and poverty status. We examined school reports of major challenges 
by performance status. These analyses did not identify clear patterns and are not part of this report. 
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a "major" challenge. For context, we also present the number or percentage of SEAs, districts, and 
schools rating the challenge (i.e., the number or percentage engaged in a relevant reform effort). 

Interpretation of results 

Several factors affect interpretation of indicator results. First, the survey asked SEAs, districts, 
and schools to self-report on their reform activity. Some respondents may have over- or under-stated 
their status in implementing reforms promoted by the act. Second, the surveys relied on closed-ended 
questions to ask about particular reform policies, programs, strategies, and practices. It is possible that 
SEAs, districts, and schools may have been working on a reform in a way not captured by our survey 
questions. Third, the indicators represent a high-level snapshot of SEA, district, and school response to 
Recovery Act reform priorities and do not measure the intensity or quality of reform efforts. The surveys 
were designed to collect information about many different reform activities. Because the survey asked 
about a wide range of activities, SEAs, districts, and schools were asked to indicate whether any activity 
occurred in the given year, rather than describe the nature of each activity in detail. The first factor may 
limit the validity of some results, while the second two factors suggest caution in drawing conclusions 
about the depth or quality of reform efforts. For example, two states or districts might have responded 
that they had supported or implemented training for educators on new standards, but the quality and 
duration of that training could have differed substantially. 

Report Contents 

The report is organized by the four reform assurance areas, with a separate chapter for each 
area: standards and assessments (chapter 2), data systems (chapter 3), educator workforce 
development (chapter 4), and support for low-performing schools (chapter 5). Within each chapter, we 
discuss the following at the state level first, then again at the district and school levels: 

• Context for each assurance area, including previous federal reform efforts, new initiatives or 
augmentation of existing initiatives brought about by the Recovery Act, and a description of the 
specific reform implementation indicators we examined. 

• Implementation in 2011-12 : the number of SEAs and percentage of districts and schools meeting 
each indicator. 

• Progress from 2009-10 to 2011-12 : the number of SEAs and percentage of districts and schools 
that were already implementing a reform in 2009-10 compared with the number and 
percentage implementing in 2010-11 and 2011-12. 

• Implementation in 2011-12 by relevant characteristics : the percentage of SEAs, districts, and 
schools meeting each indicator by several characteristics to determine if certain types of entities 
were more likely to implement reforms than others. 

• Challenges reported in 2011-12 : the number of SEAs and percentage of districts and schools 
reporting major implementation challenges within each reform area. 
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The last section of each chapter includes a cross-level comparison of the progress of reform 
implementation and challenges in 2011-12. The final chapter (chapter 6) provides an overview of state, 
district, and school reform implementation across the four assurance areas; and includes an analysis of 
state reports of major challenges by RTT status. 
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Chapter 2: Standards and Assessments 


The Recovery Act cited the state-level adoption and implementation of rigorous college- and 
career-ready standards and aligned high-quality assessments as pivotal steps in accelerating educational 
improvement throughout the United States. To facilitate these steps, the act authorized states to use 
appropriated funds to advance reforms in these areas. As a condition for receipt of SFSF funds, the 
Recovery Act held states accountable for improving state academic standards and enhancing the quality 
of academic assessments. 

State content standards have been a central priority of ESEA since its 1994 reauthorization 
which required states to establish statewide standards in reading and mathematics in selected grades 
and to implement statewide assessments and accountability systems for evaluating school-level 
performance. No Child Left Behind (NCLB) extended and strengthened this priority by requiring 
standards and assessments in grades and subjects not previously covered under the 1994 ESEA and by 
establishing additional rules for state accountability systems. In the wake of these policies, however, the 
content and rigor of standards and assessments continued to vary among states. In addition, as the 
Secretary of Education stated in 2010, "we need to raise our standards so that all students are 
graduating prepared to succeed in college and the workplace." 42 To improve the measurement of 
student achievement and to help educators improve instruction, ED also cited the need for improved 
student assessments aligned to rigorous standards (U.S. Department of Education, 2010b). 

With a priority on adoption of common standards focused on college- and career-readiness and 
common assessments, the Recovery Act and the RTT criteria signaled support for state collaboration. 
One articulation of these policies, the final SFSF rules issued in November 2009, encouraged "states to 
work together to develop and implement common, internally benchmarked standards and assessments 
aligned to those standards, in order to ensure that students are college- and career-ready." While the 
act did not explicitly endorse the CCSS, new policies established incentives for states to adopt the CCSS 
in reading/English language arts and mathematics. The CCSS were developed by the National Governors 
Association, the Council of Chief State School Officers, and other national organizations as college- and 
career-readiness standards and released in June 2010. The CCSS are grade-by-grade standards intended 
to: (1) set the same rigorous standards for all students, regardless of where they live; (2) align standards 
to the expectations of higher education and the 21st century workplace; (3) enable parents, educators, 
and policymakers to track the progress of students in meeting college- and career-ready standards at 
each grade throughout elementary and secondary schooling; (4) provide guidance for instructional 
practice, the design of curricula and instructional materials, professional development, and the content 
of teacher education; and (5) provide the basis for evaluating and holding students, teachers, schools, 
and school districts accountable for student learning (Common Core State Standards Initiative, 2012). 
Adoption of common standards also was intended to facilitate states working together to develop 
common assessments and to update those standards as needed over time, saving states time and 
money and reducing redundancy and inconsistencies across states. 

Under the Recovery Act, ED funded two multi-state consortia, the Partnership for Assessment of 
Readiness for College and Careers (PARCC) and the Smarter Balanced Assessment Consortium (SBAC), to 
develop the next generation of K-12 assessments for the general student population. These assessments 
are designed to align with the CCSS, produce common measurements of student growth and 


U.S. Department of Education, 2010a. 
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performance across states, and support state and local accountability for the CCSS. The consortium- 
developed tests are expected to be in place in the 2014-15 or 2015-16 school years. 


The Recovery Act programs also provided incentives for school districts and others to support 
the transition to the new standards and aligned assessments. School districts and partnerships of 
districts or schools with nonprofit organizations could apply directly for i3 grants. One of the priority 
areas for the grants was to fund innovations that complement the implementation of high standards 
and quality assessments. As part of the RTT grant selection criteria, participating districts were to 
collaborate with their SEAs to develop a high-quality plan to support the transition to college- and 
career-ready standards and aligned assessments. 

When implemented, the CCSS and aligned assessments were expected to constitute a significant 
change in educational approach and accountability for most states. Implementation of these standards 
and aligned assessments requires actions at the state, district, and school levels. To provide a picture of 
the prevalence and progress of the reforms promoted by the act, we identified and examined indicators 
of reform at all three levels. 

Using these indicators, this chapter describes the state, district, and school adoption and 
implementation of improved educational standards and assessments, including the provision of 
applicable tools and supports for implementing reform. We also report on the major challenges to 
implementing these reforms as perceived by survey respondents at each level. We also examine 
whether the prevalence of reform during the final year of the study varied by key state, district, and 
school characteristics. For states, we compare implementation by those states awarded RTT grants and 
all other states. For districts, we compare reform implementation by district poverty status and district 
enrollment size. For schools, we compare those that are low performing with all other schools. We 
conclude the chapter with a comparison of progress and challenges across levels. 


Key Findings Across Levels 

• More SEAs and higher percentages of districts and schools in CCSS states 
supported implementation of new standards by providing professional 
development and curriculum assistance in 2011-12 when compared with 
2010-11 (figures 2.1, 2.3, and 2.6). 

• A lack of curriculum and instructional materials aligned with new 
standards was a prevailing challenge across SEAs, districts, and schools in 
CCSS states in 2011-12 (tables 2.1, 2.2, and 2.3). This challenge was 
reported most frequently or second most frequently across the three 
levels. 

• An area with few reports of major challenges in 2011-12 was the concerns 
or possible opposition of staff, staff unions, parents, or others regarding 
the adoption and implementation of improved standards and 
assessments. 
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State-Level Findings 


In this section, we first describe the study's state-level indicators of reform in the improvement 
of standards and assessments. We then use the indicators to describe SEAs' implementation of these 
reforms in 2011-12 and progress from 2010-11 to 2011-12. 43 Next, we report on whether a greater 
percentage of RTT states implemented the reforms in 2011-12 compared with states that did not 
receive RTT grants. Last, we conclude this section with a report of the major challenges in implementing 
new or revised state standards and aligned assessments, as reported in 2011-12 by SEAs in states that 
had adopted the CCSS. 

State-Level Reform Indicators 

Traditionally, states have responsibility for establishing and implementing content standards. 
States also have typically facilitated district implementation of standards and assessments, using tools 
such as professional development. 

Given these roles and the act's priorities on the adoption and implementation of new or revised 
content standards and high-quality, aligned assessments, we examined four indicators of reform at the 
state level. These are whether the state: 

• Adopted the CCSS in mathematics and reading/English language arts; 

• Was a member of a federally funded consortium developing assessments aligned to the CCSS; 

• Provided, guided, or funded professional development on the CCSS; and 

• Provided instructional materials or curriculum assistance for the CCSS. 

We included these specific indicators because the SFSF required states receiving funds to take 
steps to improve the state academic-content and student-achievement standards. In addition, there 
were requirements and incentives in the RTT selection criteria for adopting and supporting the 
implementation of standards like those of the CCSS and their related assessments. For example, the RTT 
criteria included a measure of the extent to which the state was committed to improving the quality of 
its assessments, as demonstrated by whether the state was participating in a consortium of states 
formed to design and implement common, high-quality assessments. Aligned tests were needed 
because existing state tests would not necessarily be capable of assessing student mastery of the CCSS. 
The RTT selection criteria also encouraged the development of plans, in collaboration with districts, for 
the delivery of professional development and high-quality instructional materials to support the 
transition to and implementation of new standards and assessments. See appendix B for the 
components, decision rules, and specific Recovery Act requirements embodied in each indicator. 


No data are reported for 2009-10 because the CCSS were not yet available. 
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Implementation of Standards and Assessments Reforms: 2011-12 

• Forty-six SEAs reported that their states had adopted the CCSS in mathematics and 
reading/English language arts by 2011-12 (figure 2.1). 

• All but two of these states (44) were members of one of the two federally funded consortia 
developing assessments aligned to the CCSS in 2011-12. 


• More than 90 percent of the SEAs in states that adopted the CCSS reported supporting the 
implementation of the CCSS by providing professional development or instructional materials 
or curriculum assistance in 2011-12. 

o Forty-five of 46 SEAs provided, guided, or funded professional development on the CCSS for 
the benefit of districts and schools. 

o Forty-two of 46 SEAs provided instructional materials or curriculum assistance for the CCSS. 

Progress of Standards and Assessments Reforms: 2010-11 to 2011-12 

• State adoption of the CCSS and membership in a federally funded consortium developing 
aligned assessments increased between 2010-11 and 2011-12 (figure 2-1). 

o The number of SEAs reporting that they adopted the CCSS in mathematics and 
reading/English language arts increased from 43 to 46 SEAs. 

o The number of SEAs reporting that that they were members of a federally funded 
consortium developing assessments aligned to the CCSS increased from 43 to 44 SEAs. 

• The school years 2010-11 through 2011-12 saw an increase in the number of SEAs reporting 
that they supported CCSS implementation through the provision of professional development 
or of instructional materials or curriculum assistance. 

o Eight more SEAs reported that they had provided, guided, or funded professional 
development on the CCSS (increased from 37 to 45 SEAs). 

o Thirteen more SEAs reported that they had provided instructional materials or curriculum 
assistance for the CCSS (increased from 29 to 42 SEAs). 
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Figure 2-1. Number of state education agencies (SEAs) that implemented standards and assessments 
reforms: 2010-11 and 2011-12 


State had adopted the CCSS in 
mathematics and reading/English 
language arts 


State was a member of a federally 
funded consortium developing 
assessments aligned to the CCSS 


SEA provided, guided, or funded 
professional development on the 
CCSS 1 


SEA provided instructional 
materials or curriculum assistance 
for the CCSS 1 


2010-11 

2011-12 
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Figure Reads: Forty-three SEAs reported that in 2010-11 they had adopted Common Core State Standards (CCSS) for both 
mathematics and reading/English language arts. Forty-six SEAs did so by 2011-12. 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in 2010-11, the SEA is counted as 
meeting the indicator in 2011-12. See appendix A for more information about how data for previous years were used in these 
indicators. 

Notes: Respondents include 50 states and DC. No data are reported for 2009-10 because the CCSS were not yet available. 
Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 State Education Agency Surveys. 
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Implementation of Standards and Assessments Reforms by State RTT Status: 2011-12 

Because of incentives in the RTT program encouraging states to adopt the CCSS and to 
participate in the development of aligned assessments, we expected that by 2011-12, a higher 
percentage of states that had won RTT grants would have adopted the standards and participated in the 
assessment consortia compared with other states. In addition, the resources available through the RTT 
grants could have helped to finance SEAs' provision of professional development and curriculum 
supports for effective implementation of these reforms. 

• All 12 SEAs in RTT states reported in 2011-12 that their state had adopted the CCSS standards 
in mathematics and reading/English language arts, and all were members of a federally 
funded consortium developing assessments aligned to the CCSS (figure 2-2). 

• In addition, all SEAs in RTT states reported in 2011-12 that they provided, guided, or funded 
professional development on the CCSS and that they provided instructional materials or 
curriculum assistance for the CCSS. 

• While the high participation rates of RTT states in these reforms is in line with expectations, it 
is noteworthy that most of the non-RTT states also were implementing these reforms in 2011- 
12 . 

o Thirty-four of the 39 SEAs in non-RTT states reported that their state had adopted the CCSS 
in mathematics and reading/English language arts. 

o Thirty-two of the 34 SEAs in non-RTT states that had adopted the CCSS reported that their 
state was a member of a federally funded consortium developing assessments aligned with 
the CCSS. 

o Thirty-three of the 34 SEAs in non-RTT states that had adopted the CCSS provided, guided, 
or funded professional development on the CCSS. 

o Thirty of the 34 SEAs in non-RTT states that had adopted the CCSS provided instructional 
materials or curriculum assistance for the CCSS. 
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Figure 2-2. Comparison of the implementation of standards and assessments reforms in Race to the 
Top (RTT) states and in other states: 2011-12 
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Figure Reads: All 12 state education agencies (SEAs) in RTT states reported that in 2011-12 they had adopted Common Core 
State Standards (CCSS) for both mathematics and reading/English language arts (100 percent). In contrast, 34 of 39 of SEAs in all 
other had adopted CCSS for both mathematics and reading/English language arts (87 percent). 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in 2010-11, the SEA is counted as 
meeting the indicator in 2011-12. See appendix A for more information about how data for previous years were used in these 
indicators. 

2 The denominator for the Other States percentage includes only the 34 non-RTT states that adopted the CCSS in mathematics 
and reading/English language arts. 

Notes: Respondents include 50 states and DC. RTT states are the 12 states awarded RTT grants in the first two rounds of 
competition (DE, DC, FL, GA, HI, MA, MD, NC, NY, OH, Rl, TN). 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey. 
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Challenges Associated With Implementing Standards and Assessments Reforms: 
2011-12 

SEAs in states that had adopted the CCSS and were working to implement specific reform 
strategies in the area of standards and assessments were asked to report on the challenges associated 
with this work. Given their historical role in guiding standards-based reform, as discussed earlier, we 
were especially interested in challenges related to the development of and support for new aligned 
assessments and instructional materials. As some states had faced opposition to new standards and 
assessments in the past (Hadderman, 2000; Hardy, 2000), we also were interested in whether SEAs saw 
opposition to the CCSS as a major challenge. An SEA did not report on challenges related to a particular 
reform strategy if it was not implementing that strategy. 

• In 2011-12, the lack of SEA staff or expertise to provide professional development or technical 
assistance on developing interim or formative assessments to measure student mastery of the 
new or revised state content standards was the most frequent major challenge reported by 
SEAs that had adopted the CCSS (table 2-1). 

o Sixteen of the 32 SEAs that rated this challenge perceived as a major challenge their lack of 
SEA staff or expertise to provide districts with professional development or technical 
assistance on developing interim or formative assessments to measure student mastery of 
the new or revised state content standards. 

• The lack of instructional materials aligned with the new or revised state content standards 
and lack of SEA capacity to provide support for developing these materials were also reported 
as major challenges by SEAs that had adopted the CCSS and that rated these challenges in 
2011 - 12 . 

o Nineteen of the 40 SEAs that rated this challenge perceived a lack of instructional materials 
aligned with the new or revised content standards as a major challenge. 

o Sixteen of the 38 SEAs that rated this challenge perceived a lack of SEA staff or expertise to 
provide districts with professional development or technical assistance on developing 
instructional materials aligned with the new or revised state content standards as a major 
challenge. 

• No SEAs in states that adopted the CCSS reported in 2011-12 that opposition from educators 
or other groups to new or revised state content standards or state assessments was a major 
challenge. 
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Table 2-1. Number of state education agencies (SEAs) in states that adopted the Common Core 
State Standards (CCSS) that reported major challenges when implementing new or 
revised state content standards and aligned assessments: 2011-12 


SEA challenge 

Number of SEAs 
that reported 
challenge as a 
major challenge 1 

Total number of 
applicable SEAs 2 

Lack of SEA staff or expertise to provide districts with professional 
development or technical assistance on developing interim or 
formative assessments to measure student mastery of the new or 
revised state content standards 

16 

32 

Lack of instructional materials aligned with the new or revised 
state content standards 

19 

40 

Lack of SEA staff or expertise to provide districts with professional 
development or technical assistance on developing instructional 
materials aligned with the new or revised state content standards 

16 

38 

Lack of assessments to measure student mastery of the new or 
revised state content standards 

14 

35 

Lack of SEA staff or expertise to provide districts with professional 
development or technical assistance on implementing new or 
revised state content standards 

16 

43 

Lack of SEA staff or expertise to provide districts with professional 
development or technical assistance on implementing new state 
assessments 

9 

27 

Opposition from educators or other groups to the new or revised 
state content standards 

0 

43 

Opposition from educators or other groups to the new or revised 
state assessments 

0 

34 


Table Reads: In 2011-12, 16 of the 32 SEAs in CCSS states rating this challenge perceived lack of SEA staff or expertise to provide 
districts with professional development or technical assistance on developing interim or formative assessments to measure 
student mastery of the new or revised state content standards as a major challenge to implementing new or revised state 
standards and aligned assessments. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Applicable SEAs are those that rated the challenge. This excludes SEAs that identified the challenge as not applicable, SEAs 
that did not respond to the question, and SEAs that did not adopt the CCSS in both reading/English language arts and 
mathematics. 

Note: Respondents include 50 states and DC. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey. 
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District-Level Findings 


This section begins by describing the study's district-level indicators of reform in standards and 
assessments. We then use the indicators to describe districts' implementation of these reforms in 2011- 
12 and progress from 2010-11 to 2011-12. 44 Consistent with the emphasis on the CCSS at the state level, 
we focused on districts in states that had adopted the CCSS in both mathematics and reading/English 
language arts. Next, we compare districts in CCSS states on their awareness and implementation of 
these reforms in 2011-12, categorized first by district poverty status and then by enrollment size. Last, 
we report on the major challenges in implementing new or revised state standards and aligned 
assessments, as reported in 2011-12 by districts in states that had adopted the CCSS. (See appendix C 
for confidence intervals for each percentage reported in this section and the results of statistical tests to 
determine if certain reforms or challenges were more common than others in 2011-12.) 

District- Level Reform Indicators 

Districts play the central role in directing and supervising the implementation of rigorous, 
college- and career-ready standards and aligned assessments in schools and classrooms. Without 
leadership at the district level, state-adopted reforms in standards and assessments cannot drive 
improvements in classroom instruction and, hence, student achievement. Further, in order to ensure 
that their students are equipped to perform at high levels on state assessments, and to avoid state 
sanctions for poor performance, districts need to prepare teachers for teaching to the new standards 
and use aligned curricula. 

Recognizing these roles, we included four indicators of reform, all of which paralleled state-level 
indicators in this assurance area. The indicators included whether, in 2010-11 and 2011-12, the district: 

• Was aware of the state's adoption of the CCSS; 

• Provided professional development on new or revised state content standards: 

o For educators who teach or mentor mathematics or reading/English language arts, and 

o On instructional strategies for teachers to help English learners or students with disabilities 
master the content standards; and 

• Distributed instructional materials or provided selection guidance on curricula aligned with new 

or revised state content standards. 

We included the first of these indicators because district awareness of state adoption of the 
CCSS signals whether the district can be expected to have taken steps toward implementation of the 
new standards. Indeed, the pre-conditions for implementing new standards are awareness that the 
standards exist and that the district has responsibilities for implementing them. This indicator also 
measures the effectiveness of SEAs' communication with districts about the state adoption of new 
standards. 


No data are reported for 2009-10 because the CCSS were not yet available. 
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We asked districts to report on whether they provided teachers with professional development 
focused on new or revised state content standards and associated instructional strategies both for the 
general student population and also for two important student subgroups, English learners and students 
with disabilities. Consistent with the state-level indicator, we also asked districts in CCSS states to report 
on their distribution of or guidance on such materials or curricula. District provision of standards-related 
professional development and support for curriculum guidance and materials are indicators of districts' 
commitment to implementing new or revised standards in classrooms. 

Note that the district survey questions about implementation of state content standards were 
not limited to the CCSS. The surveys asked districts about the implementation of "the Common Core 
State Standards or other new or revised state content standards." Although the district analysis in this 
chapter is limited to districts in CCSS states, it is possible that districts in these states may have been 
thinking of new or revised standards other than the CCSS when answering the survey questions that 
made up the indicators. 

Implementation of Standards and Assessments Reforms: 2011-12 

• In states that had adopted the CCSS, 98 percent of districts reported in 2011-12 that they were 
aware of their state's adoption of the CCSS (figure 2.3). 

• Smaller percentages of districts (58 to 73 percent) supported the implementation of the 
standards by providing professional development or curriculum guidance/materials in 2011- 
12. 

o Seventy-three percent of districts in CCSS states reported that they provided professional 
development on new or revised state content standards for educators who teach or mentor 
mathematics or reading/English language arts. 

o Fifty-eight percent of districts in CCSS states reported that they provided professional 
development on new or revised state content standards and, specifically, on instructional 
strategies for teachers to help English learners or students with disabilities master the 
content standards. 

o Sixty-three percent of districts in CCSS states reported that they distributed instructional 
materials or provided selection guidance on curricula aligned with new or revised state 
content standards. 
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Progress of Standards and Assessments Reforms: 2010-11 to 2011-12 

• The percentage of districts that were aware of their state's adoption of the CCSS increased 
significantly from 86 percent in 2010-11 to 98 percent in 2011-12 (figure 2-3). 

• For two of the three indicators of district support for implementing new standards, the 
percentage of districts reporting support increased significantly from 2010-11 to 2011-12. 

o The percentage of districts that provided professional development on new or revised state 
content standards for educators who teach or mentor mathematics or reading/English 
language arts increased from 59 percent to 73 percent. However, there was no change in 
the percentage of districts (58 percent) providing standards-related professional 
development on instructional strategies for teachers to help English learners or students 
with disabilities. 

o The percentage of districts that distributed instructional materials or provided selection 
guidance on curricula aligned with new or revised state content standards increased from 53 
percent to 63 percent. 
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Figure 2-3. Percentage of districts in Common Core State Standards (CCSS) states that implemented 
reforms related to new or revised state standards: 2010-11 and 2011-12 


District aware of state adoption of Common 
Core State Standards 


District provided professional development on 
new or revised state content standards: 


For educators who teach or mentor 
mathematics or reading/English language 
arts 


On instructional strategies for teachers to 
help English learners or students with 
disabilities master the content standards 


District distributed instructional materials or 
provided selection guidance on curricula 
aligned with new or revised state content 
standards 



Percent of Districts in Common Core States 

§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 

Figure Reads: Eighty-six percent of districts in CCSS states reported that in 2010-11 they were aware that their state had 
adopted these standards. In 2011-12, 98 percent of districts in CCSS states were aware that their state had adopted these 
standards. The difference between the percentage for 2010-11 and the percentage for 2011-12 is statistically significant. 
Notes: The percentages in the figure are cross-sectional estimates for the population of districts in CCSS states that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this 
figure, for each year, the denominator is the estimated number of districts with sufficient data (i.e., answered enough 
questions to calculate the indicator) that are in the states that adopted the Common Core in mathematics and reading/English 
language arts as of that year. No data are reported for 2009-10 because the CCSS were not yet available. Detailed tables in 
appendix C provide confidence intervals for each percentage and the results of significance tests for comparisons of the 2011- 
12 percentages across indicators. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 District Surveys. 
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Implementation of Standards and Assessments Reforms by District Characteristics: 
2011-12 


Because of the long-standing focus of federal education policy on reducing poverty-related 
inequities, we examined the differences in reform implementation in high-poverty districts within CCSS 
states compared with other districts within CCSS states. We expected that high-poverty districts might 
be more likely than other districts to have made progress in implementing Recovery Act reforms 
because of their receipt of funding under federal programs that link funding to the incidence of poverty. 

We also assessed differences in reform awareness and implementation among districts with 
varying enrollment sizes. Our hypothesis was that districts with relatively large enrollments would over 
time have developed greater administrative capacity to support reforms such as implementing new 
standards compared with districts with smaller enrollments. 

District Poverty Status 

Statistically significant differences were found for two of the four indicators. However, in 
contrast to expectations, high-poverty districts were less likely than other districts to report reform 
activity. 


• In 2011-12, nearly all districts in CCSS states (regardless of poverty level) were aware of their 
state's adoption of these standards. Ninety-four percent of high-poverty districts compared 
with 99 percent of other districts were aware, a small but significant difference (figure 2-4). 

• Sixty-four percent of high-poverty districts in CCSS states provided professional development 
on new or revised state content standards for educators who teach or mentor mathematics or 
reading/English language arts, which is significantly lower than the 77 percent of other 
districts in CCSS states that provided this type of professional development. 

• There was no significant difference in the percentages of high-poverty and other districts that 
reported providing professional development on instructional strategies for teachers to help 
English learners or students with disabilities master the content standards or distributing 
instructional materials or curriculum guidance aligned to new or revised standards. 
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Figure 2-4. Comparison of the implementation of reforms related to new or revised state content 
standards in high-poverty and in other districts in Common Core State Standards (CCSS) 
states: 2011-12 


District aware of state adoption of 
Common Core State Standards 


District provided professional development 
on new or revised state content standards: 


For educators who teach or mentor 
mathematics or reading/English 
language arts 


On instructional strategies for 
teachers to help English learners or 
students with disabilities master the 
content standards 


District distributed instructional materials 
or provided selection guidance on curricula 
aligned with new or revised state content 
standards 



Percent of Districts in Common Core States 


*Percentage is significantly different from percentage for high-poverty districts (p < .05). 

Figure Reads: Ninety-four percent of high-poverty districts in CCSS states reported that in 2011-12, they were aware that their 
state had adopted these standards. In contrast, 99 percent of all other districts in Common Core states reported that they were 
aware that their state had adopted these standards. This difference is statistically significant. 

Notes: The percentages in the figure are cross-sectional estimates for the population of districts in CCSS states that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this 
figure, the denominator for the high-poverty percentages is the estimated number of high-poverty districts (had a child poverty 
rate above 21.66 percent) that are in states that adopted the Common Core in mathematics and reading/English language arts 
as of the 2011-12 school year and have sufficient data (i.e., answered enough questions to calculate the indicator). The 
denominator for the other percentages is the estimated number of districts with sufficient data that had a child poverty rate at 
or below 21.66 percent and are in states that adopted the Common Core as of that year. Detailed tables in appendix C provide 
confidence intervals for each percentage. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. U.S. Census Bureau. Small Area Income and Poverty 
Estimates (SAIPE) program, District Data for 2008 (USSD08.xls) for poverty data. Retrieved November 11, 2010, from 
http://www.census.eov/did/www/saipe/data/schools/index.html . 
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District Size 


• As expected, in states that had adopted the CCSS, large districts were significantly more likely 
than medium and small districts to be aware of state adoption of the CCSS and to support new 
standards with professional development and curriculum selection guidance and instructional 
materials in 2011-12 (figure 2-5). 

o While nearly all districts in CCSS states were aware of their state's adoption of these 
standards, there were small, but significant differences, by district size. One hundred 
percent of large districts (at least 50,000 students) in CCSS states were aware that their 
state had adopted the CCSS, compared with 98 percent of medium districts (15,000 to 
49,999 students) and 98 percent of small districts (14,999 students or fewer). 

o Ninety-one percent of large districts in CCSS states provided professional development on 
new or revised state content standards for educators who teach or mentor mathematics or 
reading/English language arts. In comparison, 75 percent of medium districts and 73 percent 
of small districts provided this type of professional development. 

o Seventy-four percent of large districts in CCSS states provided professional development on 
new or revised state content standards, including strategies for teachers to help English 
learners or students with disabilities master the content standards. In comparison, 61 
percent of medium districts and 58 percent of small districts provided this type of 
professional development. 

o Seventy-three percent of large districts in CCSS states distributed instructional materials or 
provided selection guidance on curricula aligned with new or revised state content 
standards. In comparison, 57 percent of medium districts and 63 percent of small districts 
took these actions. 
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Figure 2-5. Comparison of the implementation of reforms related to new or revised state content 

standards in large districts and in districts of other sizes in Common Core State Standards 
(CCSS) states: 2011-12 


District aware of state adoption of 
Common Core State Standards 

District provided professional 
development on new or revised state 
content standards: 

For educators who teach or 
mentor mathematics or 
reading/English language arts 


On instructional strategies for 
teachers to help English learners 
or students with disabilities 
master the content standards 


District distributed instructional 
materials or provided selection 
guidance on curricula aligned with new 
or revised state content standards 


Percent of Districts in Common Core States 

* Percentage is significantly different from percentage for large districts (p < .05). 

Figure Reads: One hundred percent of large districts in CCSS states reported that in 2011-12 they were aware that their state 
had adopted these standards. In contrast, 98 percent of medium and small districts in Common Core states were aware that 
their state had adopted these standards. These differences are statistically significant. 

Notes: The percentages in the figure are cross-sectional estimates for the population of districts in CCSS states that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this 
figure, the denominator for a size group is the estimated number of districts of a given size that have sufficient data (i.e., 
answered enough questions to calculate the indicator) and are in states that adopted the Common Core in mathematics and 
reading/English language arts as of the 2011-12 school year. District size is based on the student enrollment in schools in the 
district. Large districts have at least 50,000 students. Medium districts have fewer than 50,000 but at least 15,000 students. 
Small districts have fewer than 15,000 students. Detailed tables in appendix C provide confidence intervals for each percentage. 
Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. National Center for Education Statistics, Common Core of 
Data. Local Education Agency Universe Survey: School Year 2008-09 (ag081a.sas7bdat) for Enrollment Size. Retrieved August 
24, 2010, from http://nces.ed.gov/ccd/pubagency.asp . 
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Challenges Implementing Standards and Assessments Reforms: 201 1-12 


Districts in states that had adopted the CCSS reported on challenges they experienced in 
implementing reforms related to new or revised state content standards and aligned assessments. Given 
their critical role in overseeing the selection and implementation of curriculum and assessments, we 
were especially interested in challenges they faced in these areas, as well as potential opposition to the 
new standards and assessments at the local level. As with SEAs, a district did not report on challenges 
related to a particular reform strategy if it was not implementing that strategy. 

• In CCSS states, the major challenge cited by the largest percentage of districts rating the 
challenge in 2011-12 was insufficient funding to purchase new instructional materials aligned 
with new standards, as reported by 60 percent of districts (table 2-2). 

• Other major challenges cited by at least half of the districts centered on district needs for 
specific tools or resources for use in implementing the new or revised state standards and 
aligned assessments. Of the districts rating each challenge in 2011-12: 

o Fifty-seven percent reported that the lack of alignment between their current assessments 
and the new standards was a major challenge. 

o Fifty-two percent reported that insufficient funding to support instructional specialists or 
coaches to help educators implement new standards was a major challenge. 

• Concerns or opposition to new standards and assessments was reported infrequently in 2011- 
12 as a major challenge by those rating the challenge. 

o Concerns or opposition from school staff about additional assessments was reported as a 
major challenge by 22 percent of districts. 

o Concerns or opposition from school staff or staff unions to new standards was reported as a 
major challenge by 11 percent of districts. 

o Concerns or opposition from parents or other community groups to additional assessments 
was reported as a major challenge by 8 percent of districts. 

o Concerns or opposition from parents or other community groups to new standards was 
reported as a major challenge by 7 percent of districts. 
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Table 2-2. Percentage of districts in Common Core State Standards (CCSS) states that reported major 
challenges when planning or implementing new or revised state standards and aligned 
assessments: 2011-12 


District challenge 

Percent of districts 

Reported 
challenge as a 
major challenge 1 

Rating the 
challenge 2 

Insufficient funding to purchase new instructional materials aligned with new 
standards 

60 

95 

Current assessments are not aligned with the new standards 

57 

95 

Insufficient funding to support instructional specialists or coaches to help 
educators implement new standards 

52 

93 

Lack of district staff capacity or expertise to develop new curricula guides and 
instructional materials aligned with new standards 

37 

96 

Inadequate quality or availability of state-developed instructional materials 
aligned with standards 

36 

94 

Insufficient funding to provide adequate training to teachers on the content 
and use of the standards 

36 

96 

Standardized assessments not available for enough subjects or grades 

25 

96 

Lack of district staff capacity or expertise to provide guidance about or train 
educators on using new standards for their instruction 

25 

96 

Concerns or opposition from school staff about additional assessments 

22 

94 

Lack of clear state education agency (SEA) guidance or support on 
expectations concerning when and how standards should be implemented 

21 

94 

Lack of clear SEA guidance or support on specific content of new standards 

19 

93 

Lack of district staff capacity or expertise to provide guidance about or train 
educators on how to administer assessments 

12 

98 

Concerns or opposition from school staff or staff unions to new standards 

11 

87 


continued 
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Table 2-2. Percentage of districts in Common Core State Standards (CCSS) states that reported major 
challenges when planning or implementing new or revised state standards and aligned 
assessments: 2011-12 (cont'd) 


District challenge 

Percent of districts 

Reported 
challenge as a 
major challenge 1 

Rating the 
challenge 2 

Concerns or opposition from parents or other community groups to 
additional assessments 

8 

91 

Concerns or opposition from parents or other community groups to 
new standards 

7 

88 


Table Reads: In 2011-12, 60 percent of the districts rating this challenge perceived insufficient funding to purchase new 
instructional materials aligned with new standards as a major challenge to planning or implementing new or revised state 
standards. Ninety-five percent of districts in CCSS states rated this challenge. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Districts rating the challenge excludes districts that identified the challenge as not applicable, districts that did not respond to 
the question, and districts in states that did not adopt the CCSS in both reading/English language arts and mathematics. 

Notes: The percentages in the table are cross-sectional estimates for the population of districts in states that adopted the CCSS 
in mathematics and reading/English language arts that were operating in the 2010-11 school year. See appendix A for more 
information about the generalizability of the estimates. Detailed tables in appendix C provide confidence intervals for each 
percentage and the results of significance tests for comparisons of the percentages across challenges. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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School-Level Findings 


This section first describes the study's school-level indicators of reform in standards and 
assessments. We then use the indicators to describe schools' implementation of these reforms in 2011- 
12 and progress from 2010-11 to 2011-12 . 45 Consistent with the SEA and district analyses, we focused 
on schools in states that had adopted the CCSS in both mathematics and reading/English language arts. 
Next, we compare the percentage of low-performing schools in which these reforms were implemented 
to the corresponding percentages among other schools. Last, we report on major challenges related to 
planning or implementing new or revised state standards and aligned assessments, as reported in 2011- 
12 by schools in states that had adopted the CCSS. (See appendix C for confidence intervals for each 
percentage reported in this section and the results of statistical tests to determine if certain reforms or 
challenges were more common than others in 2011-12.) 

School-Level Reform Indicators 

In order for new standards to affect what students learn, teachers need to be prepared to teach 
the standards, and schools need to use an aligned curriculum. At the school level, we examined three 
indicators that reflected these requirements and that aligned with corresponding indicators at the 
district and state levels. 46 At the school level, the indicators are whether: 

• Teachers received professional development on new or revised state content standards; 

• Teachers received professional development targeted to help English learners or students with 
disabilities master new or revised state content standards; and 

• The school used curriculum or curriculum materials aligned with new or revised state content 
standards. 

Implementation of Standards and Assessments Reforms: 2011-12 

• In CCSS states, two-thirds of schools or more reported in 2011-12 that they received 
professional development on or used curriculum materials aligned to new or revised state 
standards (figure 2-6). 

o Seventy-eight percent of schools in CCSS states reported that their teachers received 
professional development on the new or revised state content standards. 

o Sixty-eight percent of schools in these states reported that their teachers received 
professional development targeted to help English learners or students with disabilities 
master new or revised state content standards. 


No data are reported for 2009-10 because the CCSS were not yet available. 

46 As with the district survey, the school survey questions about implementation of state content standards did not ask only 
about the CCSS. The survey asked schools about the implementation of "the Common Core State Standards or other new or 
revised state content standards." 
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o Sixty-six percent of schools in these states reported that the school used curriculum or 
curriculum materials aligned with new or revised state content standards. 

Progress of Standards and Assessments Reforms: 2010-11 to 2011-12 

• A significantly greater percentage of schools in CCSS states reported receiving professional 
development and used aligned curriculum in 2011-12 than in 2010-11 (figure 2-6). 

o The percentage of schools in CCSS states in which teachers received professional 

development on new or revised state content standards increased, from 63 percent to 78 
percent. 

o The percentage of schools in CCSS states in which teachers received professional 

development targeted to help English learners or students with disabilities master new or 
revised state content standards increased, from 62 percent to 68 percent. 

o The percentage of schools in CCSS states that used curriculum or curriculum materials 
aligned with new or revised state content standards increased, from 60 percent to 66 
percent. 
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Figure 2-6. Percentage of schools in Common Core State Standards (CCSS) states that implemented 
new or revised state standards: 2010-11 and 2011-12 


Teachers received professional 
development on new or revised 
state content standards 


Teachers received professional 
development targeted to help 
English learners or students with 
disabilities master new or revised 
state content standards 


School used curriculum or curriculum 
materials aligned with new or 
revised state content standards 



Percent of Schools in Common Core States 


§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 

Figure Reads: Sixty-three percent of schools in CCSS states reported that in 2010-11 their teachers received professional 
development on new or revised state content standards. In 2011-12, 78 percent of schools in CCSS states reported that their 
teachers received such professional development. The difference between the percentage for 2010-11 and the percentage for 
2011-12 is statistically significant. 

Notes: The percentages in the figure are cross-sectional estimates for the population of schools in CCSS states that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this 
figure, for each year, the denominator is the estimated number of schools with sufficient data (i.e., answered enough questions 
to calculate the indicator) that are in the states that adopted the Common Core in mathematics and reading/English language 
arts as of that year. No data are reported for 2009-10 because the CCSS were not yet available. Detailed tables in appendix C 
provide confidence intervals for each percentage and the results of significance tests for comparisons of the 2011-12 
percentages indicators. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. 
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Implementation of Standards and Assessments Reforms by School Performance Status: 
2011-12 

Improving the performance of students in low-performing schools is a major theme throughout 
the education provisions of the Recovery Act. The adoption and use of rigorous college- and career- 
ready standards and aligned assessments are central strategies used in the Recovery Act for achieving 
such improvement in low-performing schools nationwide. To assess the adoption and implementation 
of reforms related to standards and assessments by low-performing schools, we compared patterns of 
adoption and implementation in 2011-12 in low-performing schools and other schools located in CCSS 
states. 

• Teachers in low-performing schools in CCSS states were less likely to receive standards-related 
professional development than were teachers in other schools in 2011-12 (figure 2-7). 

o In 73 percent of low-performing schools in CCSS states, teachers received professional 

development on new or revised state content standards, compared with 79 percent of other 
schools in CCSS states. 

• Low-performing schools in CCSS states were also less likely to use CCSS aligned curriculum or 
curriculum materials than other schools in 2011-12. 

o Fifty-nine percent of low-performing schools used curriculum or curriculum materials 
aligned with new or revised state content standards, compared with 67 percent of other 
schools in CCSS states. 

• There was no significant difference between low-performing and other schools in CCSS states 
in the percentage of schools reporting that teachers received professional development 
targeted to help English learners, or students with disabilities, master new or revised 
standards. 
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Figure 2-7. Comparison of the implementation of new or revised state content standards in low- 

performing schools and in other schools in Common Core State Standards (CCSS) states: 
2011-12 
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* Percentage is significantly different from percentage for low-performing schools (p < .05). 

Figure Reads: Seventy-three percent of low-performing schools in CCSS states reported that in 2011-12 their teachers received 
professional development on new or revised state content standards. In contrast, 79 percent of all other schools in CCSS states 
reported that their teachers received such professional development. This difference is statistically significant. 

Notes: The percentages in the figure are cross-sectional estimates for the population of schools in CCSS states that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this 
figure, the denominator is the estimated number of schools in the performance status category (low performing, other) with 
sufficient data (i.e., answered enough questions to calculate the indicator) and in states that adopted the Common Core in 
mathematics and reading/English language arts as of the 2011-12 school year. Low-performing schools include schools that 
were (1) in improvement, corrective action, or restructuring; (2) identified as among the lowest-achieving schools; or (3) that 
have had a graduation rate below 60 percent over a number of years. Detailed tables in appendix C provide confidence intervals 
for each percentage. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. Approved state applications for School Improvement Grants 
for school performance status. Retrieved December 2010 from http://www2.ed.gov/programs/sif/ . 
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Challenges Implementing Standards and Assessments Reforms: 201 1-12 

Schools were asked about the challenges they experienced when planning or implementing the 
new or revised state content standards and assessments. The possible challenges were similar to those 
we posed to districts but modified to adapt them to the school context. As with SEAs and districts, a 
school did not report on a challenge if it was not using the relevant reform strategy. 

• The two challenges cited most frequently as major by schools in CCSS states in 2011-12 
involved insufficient funding (table 2-3). Of the schools rating the challenge: 

o Forty-three percent reported that insufficient funding to support instructional specialists or 
coaches to help teachers implement new standards was a major challenge. 

o Forty-two percent reported that insufficient funding to purchase new instructional materials 
aligned with new standards was a major challenge. 

• Small percentages of schools in CCSS states reported that concerns or opposition from staff, 
parents, or other community groups was a major challenge. Of the schools rating the 
challenge: 

o Concerns or opposition from school staff or staff unions about new standards was reported 
as a major challenge by 8 percent of schools. 

o Concerns or opposition from parents or other community groups to additional assessments 
was reported as a major challenge by 4 percent of schools. 

o Concerns or opposition from parents or other community groups to new standards was 
reported as a major challenge by 4 percent of schools. 
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Table 2-3. Percentage of schools in Common Core State Standards (CCSS) states that reported major 
challenges when planning or implementing new or revised state standards and aligned 
assessments: 2011-12 


School challenge 

Percent of schools 

Reported 
challenge as a 
major challenge 1 

Rating the 
challenge 2 

Insufficient funding to support instructional specialists or coaches to help 
teachers implement new standards 

43 

84 

Insufficient funding to purchase new instructional materials aligned with 
new standards 

42 

85 

Current assessments are not aligned with the new standards 

30 

86 

Inadequate quality or availability of state-developed instructional 
materials aligned with standards 

28 

86 

Lack of school staff or expertise to develop new curricula guides and 
instructional materials aligned with new standards 

25 

87 

Lack of school staff or expertise to provide guidance about or train 
educators on using new standards for their instruction 

20 

87 

Standardized assessments not available for enough subjects or grades 

16 

90 

Lack of clear district guidance or support on expectations concerning 
when and how standards should be implemented 

12 

86 

Concerns or opposition from school staff about additional assessments 

11 

93 

Lack of clear district guidance or support on specific content of new 
standards 

9 

86 

Concerns or opposition from school staff or staff unions about new 
standards 

8 

82 

Lack of school staff or expertise to train educators on how to administer 
assessments 

6 

93 

Concerns or opposition from parents or other community groups to 
additional assessments 

4 

90 

Concerns or opposition from parents or other community groups to new 
standards 

4 

84 


Table Reads: In 2011-12, 43 percent of schools in CCSS rating this challenge perceived insufficient funding to support 
instructional specialists or coaches to help teachers implement new standards as a major challenge to planning or 
implementing new or revised state standards. 

1 The challenge could be rated as not a challenge, as a minor challenge, or as a major challenge. 

2 Schools rating the challenge excludes schools that identified the challenge as not applicable, schools that did not respond to 
the question, and schools in states that did not adopt the CCSS in both reading/English language arts and mathematics. 

Notes: The percentages in the table are cross-sectional estimates for the population of schools in states that adopted the CCSS 
in mathematics and reading/English language arts that were operating in the 2010-11 school year. See appendix A for more 
information about the generalizability of the estimates. Detailed tables in appendix C provide confidence intervals for each 
percentage and the results of significance tests for comparisons of the percentages across challenges. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Comparisons Across Levels 


In this section, we point out similarities and differences among the findings for SEAs, districts, 
and schools in the adoption and implementation of reforms in standards and assessments and in the 
major challenges experienced at each level. 

Progress of Standards and Assessments Reforms: 2010-11 to 2011-12 

In most states, SEAs have primary responsibility for educational decisions regarding the 
adoption and implementation of changes in statewide standards and assessments. Districts and schools 
generally begin to implement after states adopt standards and provide guidance. Because of this, we 
expected that implementation might vary across the SEA, district, and school levels. 

• More SEAs and higher percentages of districts and schools in CCSS states supported 

implementation of new standards by providing professional development and curriculum 
assistance in 2011-12 when compared with 2010-11 (figures 2-1, 2-3, and 2-6). 

o The number of SEAs reporting that they provided, guided, or funded professional 

development on the CCSS increased from 37 to 45, while the percentage of districts in CCSS 
states that reported providing professional development on new or revised state content 
standards for educators who teach or mentor mathematics or reading/English language arts 
increased from 59 percent to 73 percent. The percentage of schools in CCSS states reporting 
teachers received professional development on new or revised state content standards 
increased from 63 percent to 78 percent. 

o However, while the percentage of schools in CCSS states reporting that their teachers 
received professional development targeted to help English learners or students with 
disabilities master new or revised state content standards increased from 62 to 68 percent, 
there was no significant increase in the percentage of districts reporting that they provided 
such professional development. 

o The number of SEAs reporting that they provided instructional materials or curriculum 
assistance for the CCSS increased from 29 to 42, while the percentage of districts reporting 
that they distributed instructional materials or provided guidance on curricula aligned with 
new or revised state content standards increased from 53 percent to 63 percent, and the 
percentage of schools reporting using aligned materials increased from 60 to 66 percent. 
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Challenges Associated With Implementing Standards and Assessments Reforms: 
2011-12 

Because of their different roles in educational governance and delivery, educators at state, 
district, and school levels are likely to experience somewhat different challenges in achieving sought- 
after reforms. In the area of reform in standards and assessments, however, two challenges were 
ranked very highly across the three levels of education governance. 

• A lack of curriculum and instructional materials aligned with new standards was a prevailing 
challenge across SEAs, districts, and schools in CCSS states in 2011-12 (tables 2-1, 2-2, and 
2-3). This challenge was reported most frequently or second most frequently across the three 
levels. 

o Nineteen of the 40 SEAs in CCSS states rating this challenge reported that a lack of 

instructional materials aligned with new or revised state content standards was a major 
challenge. 

o Sixty percent of districts in CCSS states reported that insufficient funding to purchase new 
instructional materials aligned with the new standards was a major challenge. 

o Forty-two percent of schools in CCSS states reported that insufficient funding to purchase 
new instructional materials aligned with new standards was a major challenge. 

• An area with few reports of major challenges in 2011-12 was the concerns or possible 
opposition of staff, staff unions, parents, or others regarding the adoption and 
implementation of improved standards and assessments. 

o No SEAs in CCSS states reported that such opposition was a major challenge. 

o Among districts in CCSS states, 22 percent or less reported that concerns or opposition from 
school staff, staff unions, parents, or other community groups about the implementation of 
new standards and assessments was a major challenge. 

o Among schools in CCSS states, 11 percent or less reported that such concerns were a major 
challenge. 
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Chapter 3: Data Systems 


Many of the Recovery Act's programs provided incentives to ensure that teachers, schools, 
districts, SEAs and other stakeholders have information about individual student outcomes, from early 
childhood through higher education and workforce entry, to drive educational improvement. 

Developing longitudinal data systems that include a set of 12 core elements, providing training on these 
systems, and encouraging data access and use were requirements or state selection criteria for the SFSF 
and RTT programs, and the supplemental Recovery Act funding for the SLDS program. These data 
systems and the access and use of their data are also key to implementing reforms in the act's other 
assurance areas. Planning and differentiating instruction, monitoring school and educator performance, 
promoting the equitable distribution of effective teachers, and evaluating the success of educational 
interventions all require collection of and access to comprehensive student as well as teacher data. 

The collection and use of student assessment data are fundamental to standards-based 
education reform, as envisioned by NCLB. These assessments are central to educational accountability, 
and state data systems provide the basic infrastructure for the use of student assessment results. The 
Education Technical Assistance Act of 2002 authorized competitive SLDS grants to enable states to 
"design, develop, and implement statewide, longitudinal data systems to efficiently and accurately 
manage, analyze, disaggregate and use individual student data, consistent with ESEA" including the 
reporting and use of student assessment data. In 2005, the U.S. Department of Education awarded its 
first round of the SLDS grants. Between 2005 and 2009, 41 states and the District of Columbia received 
an SLDS grant, and some received more than one. In 2007, the America COMPETES Act (P.L. 110-69) also 
encouraged improvements to state education data systems by identifying 12 core components for such 
systems (e.g., statewide unique student identifiers; the ability to link students with demographic, 
attendance, course taking, and performance data; the ability to link student teacher data.) 47 

Data use by educators to plan, differentiate, and assess the effectiveness of instruction is 
considered an important strategy for improving instruction and consequently student learning (Feldman 
and Tung, 2001; Fuchs, Deno, and Mirkin, 1984; Hamilton, Halverson, What Works Clearinghouse, et al., 
2009). Since 2006, the TIF program, by requiring that the achievement growth of individual teachers' 
students be used for educator evaluation and performance-based compensation, has encouraged 
districts to track student achievement growth for individual teachers. 

These data system components specified in the America COMPETES Act were the foundation of 
the Recovery Act's requirement for longitudinal data systems that support the other three assurance 
areas and were a condition for receiving SLDS grants funded by the act. States also were required to 
commit to establish data systems that include these core system components to receive funds from the 
SFSF. For RTT, states had to demonstrate the extent to which their statewide longitudinal system 
included all 12 components. The Recovery Act provided additional support to 20 states that previously 
received SLDS grants. The Recovery Act also funneled an additional $200 million to the TIF program. A 
core element for TIF-funded performance-based compensation systems is a data system that can link 


47 The 12 components include: use of unique student identifiers; student-level enrollment, demographic, and program 
participation information; exit drop out, transfer, and completion information for P-16 programs; communication with higher 
education data systems; assessing data quality, validity, and reliability; yearly test records; information on students not 
tested; teacher identification systems that allow linking to students; student-level transcript information; college readiness 
test scores; information on students' transition from high school to postsecondary institutions; and other information to 
determine alignment and preparedness for success in postsecondary education. 
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student achievement data to teacher and principal payroll and human resources systems. TIF allowed 
grant funds to be used for developing these data systems. 


In addition to building comprehensive data systems, the Recovery Act programs encouraged 
states to promote data access and use and included incentives for districts and schools to use data. The 
goal of data access and use also build on prior initiatives, including SLDS and TIF. States applying for RTT 
were required to demonstrate how they would make statewide longitudinal data accessible to key 
stakeholders and how the state would support districts in using data to improve instruction. At the 
district and school levels, Recovery Act programs also provided new incentives for some districts and 
schools to track and use data, especially student assessment results. For example, to receive a share of 
the $3 billion Recovery Act funds for SIG, districts had to commit to implementing specific intervention 
models in their lowest-performing schools. Two of the four SIG school intervention models included the 
use of student data to inform and differentiate instruction, and encouraging evaluation of educators 
based on student growth is part of a third SIG model. 

Taking the strategy of using data in instructional decisions to scale depends on providing 
educators with access to assessment results and training educators in assessment data interpretation 
and use, which are both actions promoted by the Recovery Act. To provide a picture of the prevalence 
and progress of the data system reforms promoted by the act, we identified and examined indicators of 
reform at the state, district, and school levels. 


Key Findings Across Levels 

• While more SEAs facilitated access to assessment data and provided 
professional development or technical assistance on their use in 2011-12 
than 2009-10, there was no corresponding district trend. However, a 
significantly larger percentage of schools reported in 2011-12 than in 
2009-10 that teachers had online access to student assessment results and 
used student assessment data to identify students for additional support 
or to tailor instruction (figures 3.1, 3.3, 3.6). 

• At both the district and school levels, the two challenges most frequently 
rated as major were delays in transmission of assessment results to 
schools or teachers (rated as a major challenge by 35 percent of districts 
and 21 percent of schools) and insufficient funding to support data 
systems that store and provide access to assessment information (rated as 
a major challenge by 33 percent of districts and 18 percent of schools) 
(tables 3.2 and 3.3). 
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Using these indicators, this chapter describes the prevalence and progress of state, district, and 
school implementation or support for implementation of Recovery Act reforms aimed at the 
development, training, and use of data systems. We also report on the major challenges to 
implementing these reforms as perceived by survey respondents at each level. We also examine 
whether the prevalence of reform during the final year of the study varied by key state, district, and 
school characteristics. For states, we compare implementation by those states awarded RTT grants and 
all other states. For districts, we compare reform implementation by district poverty status and district 
enrollment size. For schools, we compare those that are low performing with all other schools. We 
conclude the chapter with a comparison of progress across levels. 48 


State-Level Findings 


In this section, we first describe the study's state-level indicators of reform related to data 
systems. We then use the indicators to describe SEAs' implementation of these reforms in 2011-12 and 
progress from 2009-10 to 2011-12. Next, we report on whether a greater percentage of RTT states 
implemented data system reforms in 2011-12 compared with states that did not receive RTT grants. 

Last, we conclude this section with a report of the major challenges using data to support reform, as 
reported in 2011-12 by SEAs. 

State-Level Reform Indicators 

Given states' leadership in designing and implementing data systems, the act's emphasis on the 
importance of robust data systems, and specific programs promoting data access and use, we examined 
four indicators at the state level. These are whether: 

• The state operated a longitudinal data system that contained the 12 core components of a 
longitudinal education data system identified in the America COMPETES Act; 

• The state's longitudinal data system had the ability to link teachers to student data; 

• The SEA facilitated educators' access to assessment data; and 

• The SEA provided professional development or technical assistance to support educators' use of 
assessment data. 

We included the first two indicators because, as noted above, a data system including all 12 
components identified in the America COMPETES Act was one of the selection criteria in the RTT 
competition, and states were required by SFSF to report on which of these components were included in 
their longitudinal data systems. Particularly important among the 12 components is the capacity to link 
teacher and student data. The linking of these data is a prerequisite for using student growth to evaluate 
teachers and provide performance-based compensation, as promoted by the provisions of RTT and TIF 
intended to improve the quality of the educator workforce. 


48 This chapter does not include a comparison of challenges across the SEA, district, and school levels as there are no directly 
comparable challenges common to all three levels. 
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We included the last two indicators because data access and professional development on its 
use are the first steps in helping educators make use of student data. In addition, RTT selection criteria 
included having plans for supporting LEAs and schools in providing educators with information they 
need to inform and improve their instructional practices, decision-making, and overall effectiveness. The 
selection criteria also included having plans for using data by providing professional development to 
teachers, principals, and administrators on how to use data systems and the data they contain to 
support instructional improvement. See appendix B for the components, decision rules, and specific 
Recovery Act requirements embodied in each indicator. 

Implementation of Data Systems Reforms: 2011-12 

• In 2011-12, most SEAs reported that they provided educators with assistance in data access 
and use (45 to 47 SEAs, figure 3-1). 

o Forty-five SEAs reported that they facilitated educators' access to assessment data. 

o Forty-seven SEAs reported that they provided professional development or technical 
assistance to support educators' use of assessment data. 

• The latest data available show that in 2010-11, 30 SEAs reported that their state data systems 
had the ability to link teachers to student data, and 13 reported having a longitudinal data 
system that contained the 12 core components of a longitudinal education data system as 
identified in the America COMPETES Act. 49 

Progress of Data Systems Reforms: 2009-10 to 2011-12 

• Many SEAs reported that they were already providing assistance in data access (35 states) and 
use (32 states) in 2009-10, and numbers increased from 2009-10 to 2011-12 (figure 3-1). 50 

o Ten more SEAs reported that they facilitated educators' access to assessment data 
(increased from 35 to 45 SEAs). 

o Fifteen more SEAs reported that they provided professional development or technical 
assistance to support educators' use of assessment data (increased from 32 to 47 SEAs). 


Data about the status of these indicators in 2011-12 were not available from the U.S. Department of Education. 

50 Because the indicator of states providing professional development or technical assistance measures activities that may 
remain in effect across years, SEAs that reported reforms in this area in either of the previous years were considered as 
continuing the activity in 2011-12. 
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Figure 3-1. Number of state education agencies (SEAs) that implemented data system reforms: 2009- 
10, 2010-11, and 2011-12 


SEA facilitated educators' access to 
assessment data 


SEA provided professional 
development or technical assistance 
to support educators' use of 
assessment data 1 


State data system had ability to link 
teachers to student data 


State operated a longitudinal data 
system that included 12 core 
components 



Number of SEAs 

Figure Reads: Thirty-five SEAs reported that in 2009-10 they facilitated educators' access to assessment data. Forty-three SEAs 
did so in 2010-11, and 45 did so in 2011-12. 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in a previous year, the state is 
counted as meeting the indicator in subsequent years. See appendix A for more information about how data for previous 
years were used in these indicators. 

Note: Respondents include 50 states and DC. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 State Education Agency Surveys for information on 
educators' access to assessment data and professional development or technical assistance to support the use of assessment 
data. U.S. Department of Education, State Fiscal Stabilization Fund Initial Annual State Reports (2009-10) and Amended 
Applications (2010-11) for information on comprehensive data systems. States' status on the core components for 2011-12 was 
not available from the U.S. Department of Education. 
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Implementation of Data Systems Reforms by State RTT Status: 2011-12 

Because the RTT program provided additional incentives and resources for states to develop 
data systems and to plan for data access and use by educators, we expected that by 2011-12, a higher 
percentage of states that had won RTT grants would have implemented the data systems reforms we 
examined compared with other states. As noted previously, this report does not examine the 
relationship between amount of Recovery Act funding (including RTT funding) and reform 
implementation. 

• Although high percentages of non-RTT states (87 to 90 percent)implemented these reforms in 
2011-12, SEAs in RTT-winning states, as expected, were more likely than SEAs in other states 
to assist in educators' data access and use (figure 3-2). 

o Eleven of the 12 SEAs in RTT-winning states reported that they facilitated educators' access 
to assessment data compared with 34 of the 39 SEAs in the other states. 

o All 12 SEAs in RTT-winning states reported that they provided professional development or 
technical assistance to support educators' use of assessment data compared with 35 of the 
39 SEAs in other states. 
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Figure 3-2. Comparison of the implementation of data system reforms in Race to the Top (RTT) states 
and in other states: 2011-12 
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Figure Reads: Eleven of the 12 SEAs in RTT states reported that in 2011-12 they facilitated educators' access to assessment data 
(92 percent). In contrast, 34 of 39 SEAs in other states had provided such access (87 percent). 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in a previous year, the state is 
counted as meeting the indicator in subsequent years. See appendix A for more information about how data for previous 
years were used in these indicators. 

Notes: Respondents include 50 states and DC. RTT states are the 12 states awarded RTT grants in the first two rounds of 
competition (DE, DC, FL, GA, HI, MA, MD, NC, NY, OH, Rl, TN). 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey. States' state status on the core components 
for 2011-12 was not available from the U.S. Department of Education. 
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Challenges Associated With Implementing Data Systems Reforms: 2011-12 

SEAs were asked about challenges they encountered in linking students and teachers, providing 
access to, and supporting educators' use of data. An SEA did not report on challenges related to a 
particular reform strategy if it was not implementing that strategy. 


• Challenges related to linking student data to individual teachers were the most frequent 
major challenges reported by SEAs in 2011-12 (table 3-1). 

o Fourteen of the 42 SEAs rating this challenge perceived restrictions in rules and regulations 
on linking of student data to individual teachers as a major challenge. 

o Fifteen of the 48 SEAs rating this challenge perceived that current data systems make linking 
student test data to individual teachers difficult as a major challenge. 

• The challenge least frequently reported as major was that current data systems limit LEA and 
school access to new assessment data. 

o Seven of the 38 SEAs rating this challenge perceived it as a major challenge. 
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Table 3-1. Number of state education agencies (SEAs) that reported major challenges using data to 
support reform: 2011-12 


SEA challenge 

Number of SEAs 
that reported 
challenge as a 
major challenge 1 

Total number of 
applicable SEAs 2 

Restrictions in rules and regulations on linking of student data to 
individual teachers 

14 

42 

Current data systems make: 



Linking student test data to individual teachers difficult 

15 

48 

Tracking the success of school improvement efforts at the student 
level difficult 

12 

51 

Current data systems limit LEA and school access to new 
assessment data 

7 

38 

Lack of SEA staff or expertise to provide districts with professional 
development or technical assistance on accessing and using assessment 
data 

9 

42 


Table Reads: In 2011-12, 14 of the 42 SEAs rating this challenge perceived restrictions in rules and regulations on linking of 
student data to individual teachers as a major challenge to using data to support reform. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Applicable SEAs are those that rated the challenge. This excludes SEAs that identified the challenge as not applicable and SEAs 
that did not respond to the question. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey. 
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District-Level Findings 


This section begins by describing the study's district-level indicators of reform related to 
educators' access to and use of student assessment data and the use of assessment data to track 
student achievement gains for individual teachers. We then use the indicators to describe districts' 
implementation of these reforms in 2011-12 and progress from 2009-10 to 2011-12. Next, we compare 
the percentage of districts implementing these reforms in 2011-12, first by district poverty level and 
then by enrollment size. Last, we conclude this section with a report of the major challenges related to 
access and use of assessment data, as reported in 2011-12 by districts. (See appendix D for confidence 
intervals for each percentage reported in this section and the results of statistical tests to determine if 
certain reforms or challenges were more common than others in 2011-12.) 

District Reform Indicators 

Districts, as employers of educators, are likely to control access to student data and be primarily 
responsible for training educators to use it. The RTT program and Recovery Act funds for TIF provided 
incentives to promote data use within districts. At the district level, we examined three indicators of 
reform related to the data systems priorities of the Recovery Act: 

• Whether the district provided educators with access to assessment data; 

• Whether the district provided educators with professional development on the use of 
assessment data for instructional planning; and 

• Whether the district used longitudinal data to track student achievement gains for individual 
teachers. 

We included the first two indicators because RTT selection criteria envisioned that states and districts 
would partner to provide educators with information to improve instruction and professional 
development on how to use data systems and data. We included whether districts used longitudinal 
student assessment data to track student achievement gains for individual teachers because districts are 
in the key position to use this information for personnel decisions, for designing professional 
development, and for assessing the impact of initiatives aimed at improving instruction. The TIF program 
also required districts to use longitudinal data to measure student growth for use in teacher evaluation 
systems. 
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Implementation of Data Systems Reforms: 2011-12 

• At least 80 percent of districts reported that in 2011-12 they provided educators with access 
to assessment data and assistance with data use (figure 3-3). 

o Eighty-nine percent of districts reported that they provided educators with access to 
assessment data. 

o Eighty percent of districts reported that they provided educators with professional 
development on the use of assessment data for instructional planning. 

• Sixty-six percent of districts reported that they used longitudinal data to track student 
achievement gains for individual teachers in 2011-12. 

Progress of Data Systems Reforms: 2009-10 to 2011-12 

• High percentages of districts were already implementing reforms in providing educators with 
access to assessment data and professional development on data use for instructional planning 
in 2009-10, and there was no increase from 2009-10 to 2011-12. 

• For only one reform (used longitudinal data to track student achievement gains for individual 
teachers) was there a significant increase from 2009-10 to 2011-12 in the percentage of 
districts implementing (figure 3-3). 

o A higher percentage of districts used longitudinal data to track student achievement gains 
for individual teachers in 2011-12 than did so in 2009-10 (66 percent compared with 60 
percent). 
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Figure 3-3. Percentage of districts that supported reforms related to educators' use of student data: 
2009-10, 2010-11, and 2011-12 
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Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < .05). 

§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 

Figure Reads: Ninety-one percent of districts reported that they provided educators with access to assessment data in 2009-10. 
In 2010-11, 88 percent of districts reported that they provided educators with access to assessment data and 89 percent of 
districts did so in 2011-12. The difference between the percentage for 2010-11 and the percentage for 2009-10 is statistically 
significant. 

Notes: The percentages in the figure are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this figure, the 
denominator for the percentages is the estimated number of districts with sufficient data (i.e., answered enough questions to 
calculate the indicator). Detailed tables in appendix D provide confidence intervals for each percentage and the results of 
significance tests for comparisons of the 2011-12 percentages across indicators. 

For 2010-11 and 2011-12, districts were asked whether they used or included the strategy in all schools, some schools, or if 
they were not using the strategy, whether they were actively planning its use or had no plans to use or include the strategy. 
Only districts that reported that they used or included the strategy in all or some schools were counted as meeting the 
indicator. For the 2009-10 school year, the response options included only Yes and No, and only those districts that responded 
Yes were counted as meeting the indicator. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 District Surveys. 
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Implementation of Data Systems Reforms by District Characteristics: 2011-12 

Because of the long-standing focus of federal education policy on reducing poverty-related 
inequities, we examined the differences in reform implementation in high-poverty districts compared 
with other districts. We expected that high-poverty districts might be more likely than other districts to 
have made progress in implementing Recovery Act reforms because of their receipt of funding under 
federal programs that link funding to the incidence of poverty. In addition, training teachers in Title I 
schools on the use of data to inform and improve instruction was also an allowable use of additional 
Title I funds under the Recovery Act. High-poverty districts may also have more federal resources to use 
for data coordinators and data use training. 

We also examined differences in reform implementation to support data access and use by 
educators among districts with varying enrollment sizes. Our hypothesis was that districts with relatively 
large enrollments would over time have developed greater administrative capacity to support the data 
access and use reforms we examined compared with districts with smaller enrollments. In addition, 
larger districts may also have developed their own data systems independently of their SEAs. 

District Poverty Status 

• There were no statistically significant differences in the percentage of high-poverty and other 
districts supporting reforms in 2011-12 related to educators' use of data (figure 3-4). 
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Figure 3-4. Comparison of the support for reforms related to educators' use of student data in high- 
poverty districts and in other districts: 2011-12 
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Figure Reads: Eighty-eight percent of high-poverty districts and 89 percent of all other districts reported that they provided 
educators with access to assessment data in 2011-12. This difference is not statistically significant. 

Notes: The percentages in the figure are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this figure, the 
denominator for the high poverty percentages is the estimated number of high-poverty districts (had a child poverty rate above 
21.66 percent) that have sufficient data (i.e., answered enough questions to calculate the indicator). The denominator for the 
other percentages is the estimated number of districts with sufficient data that had a child poverty rate at or below 21.66 
percent. Detailed tables in appendix D provide confidence intervals for each percentage. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. U.S. Census Bureau. Small Area Income and Poverty 
Estimates (SAIPE) program, District Data for 2008 (USSD08.xls) for poverty data. Retrieved November 11, 2010, from 
http://www.census.gov/did/www/saipe/data/schools/index.html . 
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District Size 


• In 2011-12, large districts were significantly more likely than either medium-sized or small 

districts to support reforms related to educators' use of data (figure 3-5). 

o While nearly 90 percent of districts overall reported providing educators with access to 
assessment data, large districts were significantly more likely than other districts to report 
doing so. Ninety-nine percent of large districts (those with at least 50,000 students) 
reported that they provided educators with access to assessment data compared with 93 
percent of medium districts (15,000 to 49,999 students) and 89 percent of small districts 
(fewer than 15,000 students). 

o Ninety-four percent of large districts reported that they provided educators with 
professional development on the use of assessment data for instructional planning 
compared with 90 percent of medium districts and 79 percent of small districts. 

o Seventy-three percent of large districts reported that they used longitudinal data to track 
student achievement gains for individual teachers compared with 67 percent of medium 
districts and 66 percent of small districts. 
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Figure 3-5. Comparison of the support for reforms related to educators' use of student data in large 
districts and in districts of other sizes: 2011-12 
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*Percentage is significantly different from percentage for large districts (p < .05). 

Figure Reads: Ninety-nine percent of large districts reported that they provided educators with access to assessment data in 
2011-12. In contrast, 93 percent of medium-sized districts and 89 percent of small districts reported that they provided 
educators with access to assessment data that year. The differences between large districts and small and medium districts are 
statistically significant. 

Notes: The percentages in the figure are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this figure, the 
denominator for a size group is the estimated number of districts of a given size with sufficient data (i.e., answered enough 
questions to calculate the indicator). District size is based on the student enrollment in schools in the district. Large districts 
have at least 50,000 students. Medium districts have less than 50,000 but at least 15,000 students. Small districts have fewer 
than 15,000 students. Detailed tables in appendix D provide confidence intervals for each percentage. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. National Center for Education Statistics, Common Core of 
Data. Local Education Agency Universe Survey: School Year 2008-09 (ag081a.sas7bdat) for Enrollment Size. Retrieved August 
24, 2010, from http://nces.ed.gov/ccd/pubagency.asp . 
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District Challenges Associated With Implementing Data Systems Reforms: 2011-12 

Districts were also asked about a variety of potential challenges they were likely to encounter at 
the local level in supporting access to assessment data and data use. As with SEAs, a district did not 
report on challenges related to a particular reform strategy if it was not implementing that strategy. 

• Delays in transmission of assessment results to schools or teachers and insufficient funding to 
support data systems that store and provide access to assessment information were the 
challenges perceived as major by the largest percentage of districts in 2011-12 (table 3-2) 
rating these challenges. 

o Thirty-five percent of the districts perceived delays in transmission of assessment results to 
schools or teachers as a major challenge. 

o Thirty-three percent of districts perceived insufficient funding to support data systems that 
store and provide access to assessment information as a major challenge. 

• The challenges least likely to be perceived as major by the districts that rated them were lack 
of clear SEA guidance or support on using state assessment data systems and restrictions in 
rules and regulations relating to what can be included in state or district data systems and 
how to access them (table 3-2). 

o Both of these challenges were rated as major by 14 percent of the districts. 
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Table 3-2. Percentage of districts that reported major challenges using student assessment data: 
2011-12 



Percent of districts 

District challenge 

Reported 
challenge as a 
major challenge 1 

Rating the 
challenge 2 

Delays in transmission of assessment results to schools or teachers 

35 

94 

Insufficient funding to: 



Support data systems that store and provide access to assessment 
information 

33 

97 

Train educators in how to administer and use assessments 

26 

96 

Lack of district staff capacity or expertise to: 



Provide guidance about or train educators on how to use 
assessments to improve instruction 

19 

97 

Maintain and facilitate educators' access to assessment data 
systems 

18 

97 

Lack of clear SEA guidance or support on using state assessment data 
systems 

14 

87 

Restrictions in rules and regulations relating to what can be included in 
state or district data systems and how to access them 

14 

85 


Table Reads: In 2011-12, 35 percent of the districts rating this challenge perceived delays in transmission of assessment results 
to schools or teachers as a major challenge to using assessment data. Ninety-four percent of districts rated this challenge. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Districts rating the challenge excludes districts that identified the challenge as not applicable and districts that did not 
respond to the question. 

Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. Detailed tables in 
appendix D provide confidence intervals for each percentage and the results of significance tests for comparisons of the 
percentages across challenges. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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School-Level Findings 


This section first describes the study's school-level indicators of reform related to data use. We 
then use the indicators to describe schools' implementation of these reforms in 2011-12 and progress 
from 2009-10 to 2011-12. Next, we compare the percentage of low-performing schools in which these 
reforms were implemented to the corresponding percentages among other schools. Last, we conclude 
this section with a report of major challenges related to access and use of assessment data, as reported 
in 2011-12 by schools. (See appendix D for confidence intervals for each percentage reported in this 
section and the results of statistical tests to determine if certain reforms or challenges were more 
common than others in 2011-12.) 

School-Level Reform Indicators 

It is at the school level that teachers use assessment data to plan instruction for individual 
students, and principals use data to monitor and facilitate the instruction provided by individual 
teachers. The four data use indicators we examined at the school level were: 

• Whether teachers in the school had online access to student assessment results; 

• Whether the school used student assessment data to identify students for additional 
support; 

• Whether the school used assessment data to tailor instruction; and 

• Whether the school used longitudinal student assessment data to track student 
achievement gains for individual teachers. 

Online access to student assessment results is an important indicator because having such 
immediate access is likely to facilitate teachers' use of the information. We included whether the school 
used student assessment data to identify students for additional support and whether the school used 
assessment data to tailor instruction because these are concrete examples of ways educators can use 
the data made available by state and district data systems. 

Whether the school used longitudinal student assessment data to track student achievement 
gains for individual teachers is important to examine because in many schools, this is a potential input 
into decisions on teacher professional development, evaluation, and assignment that affect the 
equitable distribution of effective teachers. School leaders can also use these data to track the effects of 
professional development and similar initiatives to improve teaching practice and share it with teachers 
so that they can monitor the effectiveness of their instruction. Both the RTT and SIG programs 
encourage use of data by teachers and principals to improve school performance. 
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Implementation of Data Systems Reforms: 2011-12 

• In 2011-12, most schools (95 percent or greater) used student assessment data to improve 
instruction (figure 3-6). 

o Ninety-eight percent of schools reported that in 2011-12, they used student assessment 
data to identify students for additional support. 

o Ninety-five percent of schools reported that in 2011-12, they used student assessment data 
to tailor instruction. 

• Ninety-two percent of schools reported that teachers had online access to student assessment 
results in 2011-12. 

• Seventy-one percent of schools reported that in 2011-12, they used longitudinal data to track 
student achievement gains for individual teachers. 

Progress of Data Systems Reforms: 2009-10 to 2011-12 

• From 2009-10 to 2011-12, there was a statistically significant increase in the percentage of 
schools using student data to support instruction (figure 3-6). 

o A higher percentage of schools reported that they used student assessment data to identify 
students for additional support in 2011-12 than in 2009-10 (98 percent compared with 93 
percent). 

o A higher percentage of schools reported that they used student assessment data to tailor 
instruction in 2011-12 than in 2009-10 (95 percent compared with 85 percent) 

o A higher percentage of schools reported that teachers had online access to student 
assessment results (92 percent compared with 85 percent). 

o A higher percentage of schools reported that they used longitudinal data to track student 
achievement gains for individual teachers (71 percent compared with 60 percent). 
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Figure 3-6. Percentage of schools that used student data to support instruction: 2009-10, 2010-11, 
and 2011-12 
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A Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < .05). 

+ Percentage for 2011-12 is significantly different from percentage for 2009-10 (p < .05). 

Figure Reads: Ninety-three percent of schools reported that they used student assessment data in 2009-10 to identify students 
for additional support. In 2010-11, 98 percent of schools reported using student assessment data for that purpose, and 98 
percent did so in 2011-12. The difference between the percentages for 2010-11 and 2009-10 and the difference between the 
percentages for 2011-12 and 2009-10 are statistically significant. 

Notes: The percentages in the figure are cross-sectional estimates for the population of schools that were operating in the 
2010-11 school year. The denominator for these percentages is the estimated number of districts with sufficient data (i.e., 
answered enough questions to calculate the indicator). See appendix A for more information about the generalizability of the 
estimates. Detailed tables in appendix D provide confidence intervals for each percentage and the results of significance tests 
for comparisons of the 2011-12 percentages across indicators. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. 
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Implementation of Data Systems Reforms by School Performance Status: 2011-12 

Improving the performance of students in low-performing schools is a major goal of the 
Recovery Act's education provisions, and data use by low-performing schools to guide instruction and 
improvement planning is likely to be an important school improvement strategy. Use of data for 
improving instruction in low-performing schools was explicitly recognized by its inclusion in the SIG 
program's turnaround and transformation models. We therefore compared the percentage of low- 
performing schools meeting the reform indicators in 2011-12 to the percentage of other schools. 

• In 2011-12, while almost all schools reported using assessment data to identify students for 
additional support, low-performing schools were even more likely to do so than other schools 
(figure 3-7). 

o Ninety-nine percent of low-performing schools used student assessment data to identify 
students for additional support compared with 97 percent of other schools, which is a 
statistically significant difference. 

• Seventy-seven percent of low-performing schools used longitudinal data to track student 
achievement gains for individual teachers compared with 70 percent of other schools. 
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Figure 3-7. Comparison of the use of student data to support instruction in low-performing schools 
and in other schools: 2011-12 
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* Percentage is significantly different from percentage for low-performing schools (p < .05). 

Figure Reads: Ninety-nine percent of low-performing schools reported that they used student assessment data in 2011-12 to 
identify students for additional support. In contrast, 97 percent of all other schools did so. This difference is statistically 
significant. 

Notes: The percentages in the figure are cross-sectional estimates for the population of schools that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this figure, the 
denominator is the estimated number of schools in the performance status category (low performing, other) with sufficient 
data (i.e., answered enough questions to calculate the indicator). Low-performing schools include schools that were in 
improvement, corrective action, or restructuring; were identified as among the lowest-achieving schools; or that have had a 
graduation rate below 60 percent over a number of years. Detailed tables in appendix D provide confidence intervals for each 
percentage. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. Approved state applications for School Improvement grants 
for school performance status. Retrieved December 2010 from http://www2.ed.gov/programs/sif/ . 
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Challenges Associated With Implementing Data Systems Reforms: 2011-12 

Schools were asked about five of the same potential challenges in accessing and using 
assessment data as districts were asked to rate. As with SEAs and districts, a school did not report on a 
challenge if it was not using the relevant reform strategy. 

• Delays in transmission of assessment results to schools or teachers was the major challenge 
identified by the largest percentage of schools rating this challenge in 2011-12 (table 3-3). 

o Twenty-one percent of the schools rated delays in transmission of assessment results as a 
major challenge. 

• Lack of clear district guidance or support on using state and district assessment data systems 
was the challenge least likely to be perceived as major by the schools that rated this challenge 
(table 3-3). 

o Eight percent of the schools rated it as a major challenge. 
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Table 3-3. Percentage of schools that reported major challenges using data systems for storing, 
reporting, and using assessment results: 2011-12 



Percent of schools 

School challenge 

Reported 
challenge as a 
major challenge 1 

Rating the 
challenge 2 

Delays in transmission of assessment results to school or teachers 

21 

93 

Insufficient funding to purchase or sustain data systems that store and 
provide access to assessment information 

18 

90 

Lack of school staff or expertise to: 



Train educators on how to use assessments to improve instruction 

11 

94 

Maintain and facilitate educators' access to assessment data 
systems 

10 

94 

Lack of clear district guidance or support on using state and district 
assessment data systems 

8 

90 


Table Reads: In 2011-12, 21 percent of schools rating this challenge perceived delays in transmission of assessment results to 
school or teachers as a major challenge to using data systems for storing, reporting, and using assessment results. Ninety-three 
percent of schools rated this challenge. 

1 The challenge could be rated as not a challenge, as a minor challenge, or as a major challenge. 

2 Schools rating the challenge excludes schools that identified the challenge as not applicable and schools that did not respond 
to the question. 

Notes: The percentages in the table are cross-sectional estimates for the population of schools that were operating in the 2010- 
11 school year. See appendix A for more information about the generalizability of the estimates. Detailed tables in appendix D 
provide confidence intervals for each percentage and the results of significance tests for comparisons of the percentages across 
challenges. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 


69 


Comparisons Across Levels 


In this section, we point out similarities and differences across the three levels in the 
implementation of data systems reforms and major challenges experienced by districts and schools. 

Progress of Data Systems Reforms: 2009-10 to 2011-12 

• While more SEAs facilitated access to assessment data and provided professional 

development or technical assistance on their use in 2011-12 than 2009-10, there was no 
corresponding district trend. However, a significantly larger percentage of schools reported in 
2011-12 than in 2009-10 that teachers had online access to student assessment results and 
used student assessment data to identify students for additional support or to tailor 
instruction (figures 3-1, 3-3, 3-6). 

o The number of SEAs reporting that they facilitated access to assessment data and provided 
professional development or technical assistance to support educators' use of assessment 
data increased from 35 to 45 and 32 to 47, respectively, from 2009-10 to 2011-12. However, 
there was no statistically significant increase in the percentage of districts reporting that 
they provided educators with access to assessment data and professional development on 
the use of assessment data for instructional planning. 

o The percentage of schools reporting that teachers had online access to student assessment 
results increased from 85 percent in 2009-10 to 92 percent in 2011-12, the percentage of 
schools reporting that they used student assessment data to tailor instruction increased 
from 85 to 95 percent, and the percentage reporting that they used assessment data to 
identify students for additional support increased from 93 to 98 percent. 

Challenges Associated With Implementing Data Systems Reforms: 2011-12 

SEAs have a different role in implementing data systems reforms compared with districts and 
schools. As a result, the study presented a different list of potential challenges to SEAs than districts and 
schools. In this section, we present the challenges most often rated as major challenges for districts and 
schools. 


• At both the district and school levels, the two challenges most frequently rated as major were 
delays in transmission of assessment results to schools or teachers (rated as a major challenge 
by 35 percent of districts and 21 percent of schools) and insufficient funding to support data 
systems that store and provide access to assessment information (rated as a major challenge 
by 33 percent of districts and 18 percent of schools) (tables 3-2 and 3-3). 
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Chapter 4: Educator Workforce Development 


The Recovery Act provided substantial incentives for states and districts to work to increase 
teacher effectiveness and the equitable distribution of effective teachers. The act's component 
programs emphasized efforts to develop a more effective educator workforce through the preparation 
of new educators and the adoption of educator evaluation and compensation policies to promote the 
recruitment, retention, and equitable distribution of those educators who were determined to be 
effective. As part of these reforms, the Recovery Act promoted, and under some grant programs 
required, that the effectiveness of teachers and principals be, at least partially, demonstrated through 
growth in their students' achievement. 

The reform priorities of the Recovery Act built on earlier federal educator workforce policies and 
initiatives. NCLB singled out the importance of assigning "highly qualified" teachers to serve in Title I 
schools. Title II, Part A of ESEA also supported efforts to improve teacher quality by both states and 
districts. Beginning in 2006, ED's TIF program provided grants to selected SEAs and LEAs to support 
performance-based teacher and principal compensation systems. Earlier grants to states for developing 
longitudinal data systems sought to provide the infrastructure for tracking individual students' 
achievement across years, a prerequisite for assessing teachers' and principals' contribution to student 
growth. In addition, even before 2009, some states and districts were experimenting with or fully 
implementing alternative teacher preparation programs, evaluation systems, and pay structures (U.S. 
Department of Education, 2010b). For example, Florida, Kentucky, North Carolina, and Texas were early 
implementers of performance-based pay, along with districts such as Charlotte-Mecklenburg, North 
Carolina, and Denver, Colorado (Gonering, Teske, and Jupp, 2007; Heneman, Milanowski, and Kimball, 
2007; Kelley, Heneman, and Milanowski, 2002; Springer et al., 2010). 

While the reforms promoted by the Recovery Act overlapped with earlier policy and 
implementation efforts, the act provided incentives for broader implementation of these strategies 
across the country and at the state, district, and school levels. The priorities, reporting requirements, 
and incentives contained in the act's various grant programs encouraged state leadership on educator 
workforce efforts, both in areas where SEAs historically had played a strong role (e.g., teacher licensure 
and certification) and in areas where SEAs had been less involved (e.g., teacher evaluation and pay). In 
particular, the selection priorities of the RTT competitions and the SFSF requirements to report on the 
use of student growth as an evaluation criterion and on the distribution of educator performance ratings 
within districts encouraged states to pay more attention to rigorous educator evaluation. The TIF and 
SIG components of the act also extended incentives for evaluation and compensation reform directly to 
districts and schools and independent of states. TIF required both compensation differentiation based 
on student achievement gains and evaluation that was based substantially on such gains. SIG identified 
compensation differentiation based on student growth as a permissible strategy and required 
evaluation that included consideration of growth as part of the transformation model. 

Fully realizing the act's reform vision requires actions at the state, district, and school levels of 
the system. Each level has a somewhat different role, and the act's components varied in their emphasis 
on state, district, or school actions. To provide a picture of the prevalence and progress of the reforms 
promoted by the act, we identified and examined indicators of reform at all three levels. 

Using these indicators, this chapter describes the prevalence and progress of state, district, and 
school implementation or support for implementation of Recovery Act reforms aimed at improving the 
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quality of the educator workforce. We also report on the major challenges to implementing these 
reforms perceived by survey respondents at each level. In this chapter we also examine whether 
prevalence of reform during the final year of the study varied by key state, district, and school 
characteristics. For states, we compare implementation by those states awarded RTT grants and all 
other states. For districts, we compare reform implementation by district poverty status and district 
enrollment size. For schools, we compare those that are low performing with all other schools. The 
chapter ends with a comparison of the progress and challenges across levels. 


Key Findings Across Levels 

• While the number of SEAs that supported the educator workforce reforms 
related to principal evaluation, teacher evaluation, and teacher 
compensation increased between 2009-10 and 2011-12, the percentages of 
districts and schools implementing these reforms did not increase or 
decreased during this time (figures 4.1, 4.3, and 4.6). 

• Difficulty measuring student growth for teachers of subjects with no 
standardized tests was rated as a major challenge at all three levels in 
2011-12. This was the challenge most frequently rated as major by SEAs 
and was the fourth most frequently rated major challenge by districts and 
schools (tables 4.1, 4.2, and 4.3). 

• At all three levels, concerns or opposition from educators about 
performance-based compensation were more likely to be rated as a major 
challenge than concerns or opposition about evaluation based on student 
achievement in 2011-12 (tables 4.1, 4.2, and 4.3). 

• Although not asked at the SEA level, large percentages of both districts and 
schools rated insufficient funding to provide performance-based 
compensation or differential compensation as major challenges, (tables 4.2 
and 4.3). 


State-Level Findings 


In this section, we first describe the study's state-level indicators of reform related to the act's 
priorities for educator workforce improvement. We then use the indicators to describe SEAs' 
implementation of these reforms in 2011-12 and progress from 2009-10 to 2011-12. Next, we report on 
whether a greater percentage of RTT states implemented these reforms in 2011-12 compared with 
states that did not receive RTT grants. Last, this section reports on the major challenges related to 
carrying out some of the most important reforms envisioned by the act, as reported in 2011-12 by SEAs. 
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State-Level Reform Indicators 

Major components of the act such as SFSF and RTT provided incentives for states to change 
policies with respect to educator preparation, evaluation, and compensation. At the state level, we 
examined five indicators that reflect the act's priorities for educator workforce improvement. These are 
whether the SEA: 

• Simplified or shortened the educator licensure process or authorized non-university preparation 
programs; 

• Issued standards or guidelines for teacher and principal preparation programs; 

• Supported the use of multi-level ratings, multiple observations, and student achievement gains 
for teacher evaluation; 

• Supported use of student achievement gains for principal evaluation; and 

• Supported differentiating teacher compensation based on student achievement gains. 

The first indicator is included because during the last 20 years, there has been a fear of teacher 
shortages, concerns that traditional university-based programs restrict the number of teachers entering 
the market and require unnecessary coursework, and a desire to accommodate career changers who 
already have a bachelors' degree. These issues led to an interest in streamlining preparation programs 
and allowing organizations other than universities to prepare teachers (National Commission on 
Teaching and America's Future, 1996). We also included this indicator because the RTT competition 
included as a selection criterion whether states allowed alternative routes to certification for teachers 
and principals, particularly routes that allow for providers in addition to institutions of higher education, 
and whether such routes were in use. 

We also included an indicator of whether states issued standards or guidelines for teacher and 
principal preparation programs that promote alignment of teacher preparation programs to new or 
revised state content standards or state teacher performance standards, training students on practices 
to improve low-performing schools, and tracking the effectiveness of graduates. These are potential 
actions SEAs could take to improve the quality of newly prepared teachers. In addition, the RTT selection 
requirements encouraged states to link information on student growth of educators' students to the 
programs where the educators were prepared and report this information publically for each program 
approved by the state. 

Research has shown that the teacher characteristics underlying NCLB's definition of a highly 
qualified teacher are not related to teacher effectiveness (e.g., Palardy and Rumberger, 2008; Phillips, 
2010; Smith, Desimone, and Ueno, 2005), and in its proposals for ESEA reauthorization, ED proposed 
replacing teacher credentials with the results from teacher evaluations for assessing the degree to which 
effective teachers are distributed equitably (U.S. Department of Education, 2010b). We included an 
indicator of SEA support for multi-level ratings, multiple observations, and student achievement gains 
for teacher evaluation because these reflect important requirements in Recovery Act programs. Multiple 
rating levels, multiple observations, and taking student achievement gains or growth into account are 
core requirements for evaluation systems for TIF program grantees. The use of multiple levels and 
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student gains was an RTT selection criterion, and evaluation using multiple observations and student 
achievement gains was a required strategy in the SIG school transformation model. The SFSF requires 
states to report on whether teacher evaluation systems used in the state include student achievement 
outcomes or student growth data as an evaluation criterion. Teacher evaluation systems including 
measurements of student achievement growth are also argued to better distinguish between effective 
and ineffective teachers (e.g., Glazerman et al., 2010) and, if so, would provide better information to 
target efforts to improve teacher quality and its equitable distribution. States and districts can use this 
information to target professional development to less effective teachers as well as identify the most 
effective teachers from whom to learn (Garet et al., 2010; Goe, Biggers, and Croft, 2012). 

As with teachers, assessment of principal performance based on student achievement gains or 
growth has the potential for increasing the rigor of principal evaluation and providing a better measure 
of effectiveness for decisions about professional development and assignment and for assessing the 
equitable distribution of effective school leaders. We included an indicator of whether states supported 
use of student achievement gains for principal evaluation because such use was a core requirement of 
the TIF grant program, an RTT eligibility requirement and selection criterion, and because states were 
required to report whether their systems to evaluate the performance of principals include student 
achievement outcomes or student growth data as an evaluation criterion. 

We included an indicator on whether the state supported differentiating teacher compensation 
based on student achievement gains because this type of performance-based compensation was a core 
requirement of the TIF, as well as being implied by the RTT selection criterion that evaluation systems 
based on student growth would be used to inform teacher compensation decisions and that highly 
effective teachers would be provided with opportunities for additional compensation. See appendix B 
for the components, decision rules, and specific Recovery Act requirements embodied in each indicator. 

Implementation of Educator Workforce Development Reforms: 2011-12 

• Reforms related to educator preparation were the most common type of workforce reform 
implemented by 2011-12 (by 35 to 39 SEAs, figure 4-1). 

o Thirty-nine SEAs reported that they had simplified or shortened educator licensure 
processes or authorized non-university preparation programs. 51 

o Thirty-eight SEAs reported having issued standards or guidelines for teacher preparation 
programs. 

o Thirty-five SEAs reported having issued standards or guidelines for principal preparation 
programs. 

• In 2011-12, between 14 and 22 states supported reforms related to teacher and principal 
evaluation and teacher compensation. Use of achievement data for principal evaluation was 
most common among these reforms. 

o Fifteen SEAs reported that they supported differentiating teacher compensation based on 
student achievement gains. 


51 See Appendix table E.l for the number of SEAs that implemented each of these reforms. 
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o Twenty-two SEAs reported that they supported use of student achievement gains for 
principal evaluation. 

o Fourteen SEAs reported that they supported use of multi-level ratings, multiple 

observations, and student achievement gains for teacher evaluation. When we examined 
the individual evaluation components of this indicator, we observed that an additional six 
SEAs reported supporting use of student achievement gains, but did not also support the 
other two features of teacher evaluation (see appendix table E.2). 

Progress of Educator Workforce Development Reforms: 2009-10 to 2011-12 

• The number of states supporting reforms in educator preparation increased from 2009-10 to 

2011-12 52 (figure 4-1). 

o Six more SEAs reported that they simplified or shortened educator licensure processes or 
authorized non-university preparation programs (increased from 33 to 39 SEAs). 

o Ten more SEAs reported that they issued standards or guidelines for teacher preparation 
programs (increased from 28 to 38 SEAs). 

o Fifteen more SEAs reported that they issued standards or guidelines for principal 
preparation programs (increased from 20 to 35 SEAs). 

• The number of states supporting reforms in evaluation and teacher compensation increased 

from 2009-10 to 2011-12. 

o Eight more SEAs reported that they supported differentiating teacher compensation based 
on student achievement gains (increased from 7 to 15 SEAs). 

o Sixteen more SEAs reported that they supported the use of student achievement gains for 
principal evaluation (increased from 6 to 22 SEAs). 

o Thirteen more SEAs reported that they supported use of multi-level ratings, multiple 

observations, and student achievement gains for teacher evaluation (increased from 1 to 14 
SEAs). 


Because these indicators measure state activities that require a one-time action and may remain in effect across years, SEAs 
that reported reforms in either of the previous years were considered as continuing the activity in 2011-12. 
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Figure 4-1. Number of state education agencies (SEAs) that implemented educator workforce 
development reforms: 2009-10, 2010-11, and 2011-12 
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Figure Reads: Thirty-three SEAs reported that in 2009-10, they simplified or shortened the educator licensure process or 

authorized non-university educator preparation programs. Thirty-five SEAs did so by 2010-11 and 39 in 2011-12. 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in a previous year, the state is 
counted as meeting the indicator in subsequent years. See appendix A for more information about how data for previous 
years were used in these indicators. 

2 To meet these indicators, SEAs not only had to play a role in supporting implementation, but also had to be supporting the 
specific types of systems that the Recovery Act explicitly identified. For the differentiated teacher compensation indicator, 
two states that met the indicator in 2010-11 are counted as meeting the indicator in 2011-12 given that their role in 
supporting implementation was a potentially ongoing or continued activity. In addition, for the principal evaluation indicator, 
one state that met the indicator in 2010-11 is counted as meeting the indicator in 2011-12 for the same reason. See appendix 
A for more information about how data for previous years were used in these indicators. 

Notes: Respondents include 50 states and DC. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 

Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 State Education Agency Surveys. 
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Implementation of Educator Workforce Development Reforms by State RTT Status: 
2011-12 

Because of the strong emphasis of the RTT program on state-level action to improve the 
preparation and evaluation of educators, and the additional resources provided by the grants, we 
expected that by 2011-12, a higher percentage of states that had won RTT grants would have 
implemented the reforms we examined compared with other states. 

• States that were awarded RTT grants were more likely than other states to have implemented 
educator preparation reforms in 2011-12 (figure 4-2). 

o All 12 of the SEAs in RTT-wining states reported that they simplified or shortened the 
educator licensure process or authorized non-university preparation programs, compared 
with 27 of 39 SEAs in other states. Ten of the 12 SEAs in RTT-winning states issued standards 
or guidelines for principal preparation programs, compared with 25 of 39 SEAs in other 
states. 


o Nine of the 12 SEAs in RTT-winning states reported that they issued standards or guidelines 
for teacher preparation programs, compared with 29 of 39 in other states. 

• States that were awarded RTT grants were more likely than other states to have supported 
educator evaluation and compensation reforms in 2011-12. 

o The largest differences between RTT states and other states were in the area of educator 
evaluation, with RTT states being more likely to support the use of multi-level ratings, 
multiple observations, and student achievement gains for teacher evaluation (9 of 12 RTT 
states compared with 5 of the 39 other states); the use of student achievement gains for 
principal evaluation (10 of 12 compared with 12 of 39); and differentiating teacher 
compensation based on student achievement gains (6 of 12 compared with 9 of 39). 
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Figure 4-2. Comparison of the implementation of educator workforce development reforms in Race 
to the Top (RTT) states and in other states: 2011-12 
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Figure Reads: All 12 state education agencies (SEAs) in RTT states reported that in 2011-12 they had simplified or shortened the 
educator licensure process or authorized non-university educator preparation programs (100 percent). In contrast, 27 of 39 
SEAs in all other states had done so (69 percent). 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in a previous year, the state is 
counted as meeting the indicator in subsequent years. See appendix A for more information about how data for previous 
years were used in these indicators. 

2 To meet this indicator, SEAs not only had to play a role in supporting implementation, but also had to be supporting the 
specific types of systems that the Recovery Act explicitly identified. Two states that met the indicator in 2010-11 are counted 
as meeting the indicator in 2011-12 given that their role in supporting implementation was a potentially ongoing or continued 
activity. See appendix A for more information about how data for previous years were used in these indicators. 

Note: Respondents include 50 states and DC. RTT states are the 12 states awarded RTT grants in the first two rounds of 
competition (DE, DC, FL, GA, HI, MA, MD, NC, NY, OH, Rl, TN). 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey. 
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Challenges Associated With Implementing Educator Workforce Development Reforms: 
2011-12 


SEAs working to implement specific educator workforce reform strategies were asked to report 
on associated challenges. Given the Recovery Act's focus on including growth in student achievement in 
educator compensation and evaluation systems and the focus on defining "effectiveness" at least 
partially in terms of demonstrated growth in student achievement, we were especially interested in 
challenges they encountered when implementing these reforms. SEAs were also asked about challenges 
in implementing educator recruitment, hiring, and induction and in influencing educator preparation 
programs. An SEA did not report on challenges related to a particular reform strategy if it was not 
implementing a that strategy. 

• Difficulty in measuring student growth for teachers of subjects that had no standardized tests 

was the most frequent major challenge reported by SEAs in 2011-12 (table 4-1). 

o Thirty-five of the 46 SEAs rating this challenge perceived difficulty in measuring student 
growth for teachers of non-tested subjects as a major challenge. 

• Challenges related to educator compensation were more likely to be rated as major than 

similar challenges related to educator evaluation in 2011-12. 

o Twenty-four of the 34 SEAs rating this challenge rated concerns or opposition from 
educators about performance-based compensation as a major challenge. 

o Twenty-four of the 47 SEAs rating this challenge rated concerns or opposition from 

educators about evaluating educators based, at least in part, on student achievement as a 
major challenge. 

o Twelve of the 32 SEAs rating this challenge perceived restrictions in rules and regulations on 
how educators can be compensated as a major challenge. 

o Six of the 43 SEAs rating this challenge reported that restrictions in rules and regulations on 
how educators can be evaluated as a major challenge. 
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Table 4-1. Number of state education agencies (SEAs) that reported major challenges when working 
with districts and others to develop and manage a skilled educator workforce: 2011-12 


SEA challenge 

Number of SEAs 
that reported 
challenge as a 
major challenge 1 

Total number of 
applicable 
SEAs 2 

Difficulty in measuring student growth for teachers of non-tested subjects 

35 

46 

Concerns or opposition from educators about performance-based 
compensation 

24 

34 

Concerns or opposition from educators about evaluating educators based, 
at least in part, on student achievement 

24 

47 

Lack of SEA staff or expertise to provide LEAs with professional 
development or technical assistance on differentiated teacher 
compensation systems 

14 

30 

Lack of SEA staff or expertise to develop reliable and fair methods for a 
statewide system of educator performance evaluation based partly on 
student achievement 

19 

46 

Restrictions in rules and regulations on how educators can be 
compensated 

12 

32 

Lack of SEA staff or expertise to provide LEAs with professional 
development or technical assistance on educator recruitment, hiring, and 
induction 

14 

41 

Resistance from colleges and universities to modifying educator 
preparation programs to changing state reform priorities 

12 

47 

Lack of clear federal guidance or support on educator compensation or 
evaluation systems 

8 

42 

Restrictions in rules and regulations on how educators can be evaluated 

6 

43 


Table Reads: In 2011-12, 35 of the 46 SEAs rating this challenge perceived difficulty in measuring student growth for teachers of 
nontested subjects as a major challenge when working with districts and others to develop and manage a skilled educator 
workforce. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Applicable SEAs are those that rated the challenge. This excludes SEAs that identified the challenge as not applicable and SEAs 
that did not respond to the question. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey. 
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District-Level Findings 


This section begins by describing the study's district-level indicators of workforce development 
reform. We then use the indicators to describe districts' implementation of these reforms in 2011-12 
and progress from 2009-10 to 2011-12. Next, we compare the percentage of districts implementing 
these reforms in 2011-12, with districts categorized first by poverty level and then by enrollment size. 

We conclude this section with a report of the major challenges experienced by districts carrying out the 
workforce development reforms envisioned by the act, as reported in 2011-12 by districts. (See 
appendix E for confidence intervals for each percentage reported in this section and the results of 
statistical tests to determine if certain reforms or challenges were more common than others in 2011- 
12.) 

District- Level Reform Indicators 

While the Recovery Act promoted a stronger state role in improving the quality of the educator 
workforce, districts, as the employers of educators, continued to have an important influence on the 
workforce and the equitable distribution of educators. Realizing the Recovery Act's vision for improving 
the educator workforce requires that districts implement hiring, evaluation, compensation, and 
retention policies that promote educator effectiveness and the equitable distribution of teachers. In 
addition, the TIF and i3 components of the Recovery Act bypassed states by providing funding directly to 
districts willing to undertake evaluation and compensation reforms. 

At the district level, we examined three indicators of reform that parallel the state-level 
indicators and are directly related to requirements for educator evaluation or compensation in one or 
more of the act's components. 53 These are whether the district: 

• Operated a teacher evaluation system that included multi-level rubrics, multiple observations, 

and student achievement gains; 

• Operated a principal evaluation system that included student achievement gains; and 

• Differentiated teacher compensation based on student achievement gains. 

We also tracked two indicators that are specific to districts. These district-specific indicators are 
whether the district: 

• Provided school leaders with professional development or flexibility to hire effective teachers; 

and 

• Used student achievement gains for teacher tenure, dismissal, or assignment decisions. 

These two district-specific indicators were included because districts are typically responsible for 
hiring, assigning, and retaining teachers. The first of these two indicators includes two components: (a) 
providing school leaders with authority to hire more qualified transfer candidates without regard to 


53 


The rationale for these parallel indicators is provided above in the section describing the state-level reform indicators. 
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district seniority rules and (b) providing professional development to principals on recruiting and hiring 
effective teachers. The first component was included because it is often argued that seniority hiring 
provisions inhibit schools from hiring the teachers that best fit their needs (e.g., Levin, Mulhern, 
Schunck, and New Teacher Project, 2005). Providing principals with the flexibility to hire teachers from 
within the district without regard to seniority may be one way to help ensure a better match between 
the school and the teacher and preventing high-need schools from having to accept transfers of less 
effective teachers laid off from or counseled out of other schools. The second component, providing 
principals with training in hiring, helps them make better use of this flexibility, as well as helping them 
select those job applicants more likely to be effective. More informed hiring of new and transfer 
teachers can help to ensure that teachers hired are effective and to promote equitable distribution of 
effective teachers (Behrstock, Clifford, and National Comprehensive Center for Teacher Quality, 2010). 

The second district-specific indicator is based on districts' reports on whether they used student 
achievement gains for teacher tenure, dismissal, or assignment decisions. These decisions, which affect 
the equitable distribution of effective teachers and the overall quality of the teacher workforce, are 
primarily district responsibilities. Teacher effectiveness measures based on student achievement gains 
could be better indicators of teacher quality for use in making these decisions. RTT recognized this by 
requiring states to have a plan for participating LEAs to use evaluation results based on student 
achievement gains for tenure and dismissal. SFSF also required states to report on district evaluation 
systems used for these purposes. 

Implementation of Educator Workforce Development Reforms: 2011-12 

• Sixty-three percent of districts reported that in 2011-12 they provided school leaders with 
professional development or flexibility to hire effective teachers (figure 4-3). 54 

• Reforms in teacher compensation, teacher and principal evaluation, and use of student 
achievement gains for personnel decisions were less commonly reported by districts in 2011- 
12 (8 to 30 percent) than was providing school leaders with professional development or 
flexibility to hire effective teachers (figure 4-3). 

o Twenty-four percent of districts reported that they used student achievement gains for 
teacher tenure, dismissal, or assignment decisions. 

o Thirty percent of districts reported that they operated a principal evaluation system that 
included student achievement gains. 

o Seventeen percent of districts reported that they operated a teacher evaluation system that 
included multi-level rubrics, multiple observations, and student achievement gains. When 
looking at the individual evaluation components of this indicator, much higher percentages 
of districts used multiple observations (78 percent) or multi-level rubrics (69 percent) than 
used student achievement gains in NCLB grades/subjects in determining individual teacher 
performance ratings (20 percent). (See appendix table E.6) 


54 See Appendix table E.5 for the percentage of districts that implemented each of these reforms. 
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o Eight percent of districts reported that they differentiated teacher compensation based on 
student achievement gains. 

Progress of Educator Workforce Development Reforms: 2009-10 to 2011-12 

• There were no statistically significant increases in the percentage of districts that used these 
educator workforce reforms in 2011-12 than did so in 2009-10 (figure 4-3), and the 
percentages decreased significantly for three of our five indicators. 

o The percentage of districts that reported providing school leaders with professional 

development or flexibility to hire effective teachers decreased from 69 percent in 2009-10 
to 63 percent in 2011-12. 

o The percentage of districts that reported using student achievement gains for teacher 
tenure, dismissal, or assignment decisions decreased from 38 percent to 24 percent. 

o The percentage of districts that reported differentiating teacher compensation based on 
student achievement gains decreased from 12 to 8 percent. 
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Figure 4-3. Percentage of districts that implemented reforms related to educator workforce 
development: 2009-10, 2010-11, and 2011-12 
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tPercentage for 2011-12 is significantly different from percentage for 2009-10 (p < .05). 

§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 

Figure Reads: Sixty-nine percent of districts reported that in 2009-10 they provided school leaders with professional 
development or flexibility to hire effective teachers. In 2010-11, 64 percent of districts provided school leaders with 
professional development or flexibility to hire effective teachers, and 63 percent of districts did so in 2011-12. The difference 
between the percentage for 2011-12 and the percentage for 2009-10 is statistically significant. 

Notes: The percentages in the figure are cross-sectional estimates for the population of districts operating in the 2010-11 
school year. The denominator for these percentages is the estimated number of districts with sufficient data (i.e., answered 
enough questions to calculate the indicator). See appendix A for more information about the generalizability of the estimates. 
Detailed tables in appendix E provide confidence intervals for each percentage and the results of significance tests for 
comparisons of the 2011-12 percentages across indicators. 

For 2010-11 and 2011-12, districts were asked whether they used or included the strategy in all schools, some schools, or if 
they were not using the strategy; whether they were actively planning its use; or had no plans to use or include the strategy. 
Only districts that reported that they used or included the strategy in all or some schools were counted as meeting the 
indicator. For the 2009-10 school year, the response options included only Yes and No, and only those districts that responded 
Yes were counted as meeting the indicator. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 District Surveys. 
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Implementation of Educator Workforce Development Reforms by District 
Characteristics: 2011-12 

Because of the long-standing focus of federal education policy on reducing poverty-related 
inequities, we examined the differences in reform implementation in high-poverty districts compared 
with other districts. We expected that high-poverty districts might be more likely than other districts to 
have made progress in implementing Recovery Act reforms because of their receipt of funding under 
federal programs that link funding to the incidence of poverty. In addition, the TIF program also was 
restricted to schools in districts below a poverty threshold. 

We also examined differences in implementation of reforms related to educator workforce 
among districts with varying enrollment sizes. Our hypothesis was that districts with relatively large 
enrollments would over time have developed greater administrative capacity to implement these 
reforms compared with districts with smaller enrollments. In particular, larger districts may have 
invested more resources in human resources management and systems needed to evaluate educators. 

District Poverty Status 

• Contrary to expectations, in 2011-12, high-poverty districts were not more likely to implement 

workforce reforms, except for differentiating teacher compensation (figure 4-4). 

o A higher percentage of high-poverty districts than other districts reported that they 
differentiated teacher compensation based on student achievement gains (16 percent 
compared with 5 percent). 
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Figure 4-4. Comparison of the implementation of reforms related to educator workforce 
development in high-poverty districts and in other districts: 2011-12 
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^Percentage is significantly different from percentage for high-poverty districts (p < .05). 

Figure Reads: Fifty-seven percent of high-poverty districts and 65 percent of all other districts provided school leaders with 
professional development or flexibility to hire effective teachers in 2011-12 to improve the quality of teachers hired. This 
difference is not statistically significant. 

Notes: The percentages in this figure are cross-sectional estimates for the population of districts operating in the 2010-11 
school year. The denominator for the high-poverty percentages is the estimated number of high-poverty districts (had a child 
poverty rate above 21.66 percent) that have sufficient data (i.e., answered enough questions to calculate the indicator). The 
denominator for the other percentages is the estimated number of districts with sufficient data that had a child poverty rate at 
or below 21.66 percent. See appendix A for more information about the generalizability of the estimates. Detailed tables in 
appendix E provide confidence intervals for each percentage. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. U.S. Census Bureau. Small Area Income and Poverty 
Estimates (SAIPE) Program, District Data for 2008 (USSD08.xls) for poverty data. Retrieved November 11, 2010, from 
http://www.census.gov/did/www/saipe/data/schools/index.html . 
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District Size 


• As expected, in 2011-12, large districts were significantly more likely than other districts to 

report that they implemented reforms related to educator workforce development 

(figure 4-5). 

o Large districts (at least 50,000 students) were more likely than small districts (fewer than 
15,000 students) to report that they provided school leaders with professional development 
or flexibility to hire effective teachers (74 percent compared with 63 percent). However, the 
difference between large and medium districts was not significant. 

o Thirty-four percent of large districts reported that they used student achievement gains for 
teacher tenure dismissal or assignment decisions compared with 26 percent of medium 
districts (15,000 to 49,999 students) and 24 percent of small districts. 

o Fifty-nine percent of large districts reported that they operated a principal evaluation 
system that included student achievement gains compared with 50 percent of medium 
districts and 29 percent of small districts. 

o Thirty-seven percent of large districts reported that they operated a teacher evaluation 
system that included multi-level rubrics, multiple observations, and student achievement 
gains compared with 22 percent of medium districts and 16 percent of small districts. 

o Thirty-eight percent of the large districts reported that they differentiated teacher 
compensation based on student achievement gains compared with 12 percent of the 
medium-sized and 8 percent of the small districts. 
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Figure 4-5. Comparison of the implementation of reforms related to educator workforce 
development in large districts and in districts of other sizes: 2011-12 
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*Percentage is significantly different from percentage for large districts (p < .05). 

Figure Reads: Seventy-four percent of large districts provided school leaders with professional development or flexibility to hire 
effective teachers in 2011-12. In contrast, 71 percent of medium-sized districts and 63 percent of small districts did so. The 
difference between the percentages for large and small districts is statistically significant. 

Notes: The percentages in this figure are cross-sectional estimates for the population of districts operating in the 2010-11 
school year. The denominator for a size group is the estimated number of districts of a given size with sufficient data (i.e., 
answered enough questions to calculate the indicator). District size is based on the student enrollment in schools in the district. 
Large districts have at least 50,000 students. Medium districts have less than 50,000 but at least 15,000 students. Small districts 
have fewer than 15,000 students. See appendix A for more information about the generalizability of the estimates. Detailed 
tables in appendix E provide confidence intervals for each percentage. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. National Center for Education Statistics, Common Core of 
Data. Local Education Agency Universe Survey: School Year 2008-09 (ag081a.sas7bdat) for enrollment size. Retrieved August 
24, 2010, from http://nces.ed.gov/ccd/pubagency.asp . 
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Challenges Associated With Implementing Educator Workforce Development Reforms: 
2011-12 

Like SEAs, districts were also asked about a variety of potential challenges in implementing the 
reforms promoted by the act. Some of the challenges were similar to those we asked of SEAs, while 
others were more specific to district roles in implementing educator workforce reforms. For example, 
districts were likely to be responsible for the costs of performance-based compensation. Grantees 
participating in the TIF program were expected to take on greater responsibility for costs associated 
with performance bonuses. As with SEAs, a district did not report on challenges related to a particular 
reform strategy if it was not implementing that strategy. 

• Insufficient funding for teacher compensation reform was the major challenge identified by 
the largest percentage of districts rating the challenges in 2011-12 (table 4-2). 

o Eighty-four percent of districts perceived insufficient funding to provide differential 
compensation for teachers in high-need areas as a major challenge. 

o Eighty-three percent of districts perceived insufficient funding to provide performance- 
based compensation to all eligible teachers as a major challenge. 

• Fifty-nine percent of districts perceived difficulty in measuring student growth for teachers of 
nontested subjects as a major challenge in 2011-12. 

• Challenges related to educator compensation were more likely to be rated as major than 
similar challenges related to educator evaluation in 2011-12 

o Concern or opposition from educators related to performance-based compensation was 
rated as a major challenge for 59 percent of the districts, while 50 percent rated concerns or 
opposition about evaluating educators based on student achievement as a major challenge. 

o Fifty-six percent of districts rated restrictions in rules and regulations on how educators can 
be compensated as a major challenge, while only 36 percent rated restrictions in rules and 
regulations on how educators can be evaluated as a major challenge. 
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Table 4-2. Percentage of districts that reported major challenges when implementing educator 
evaluation and compensation systems: 2011-12 


District challenge 

Percent of districts 

Reported 
challenge as a 
major challenge 1 

Rating the 
challenge 2 

Insufficient funding to provide differential compensation for teachers in high- 
need areas (e.g., low-performing schools, science, technology, engineering, 
and mathematics subjects) 

84 

53 

Insufficient funding to provide performance-based compensation to all 
eligible teachers 

83 

57 

Concerns or opposition from school staff or staff unions about performance- 
based compensation 

59 

73 

Difficulty in measuring student growth for teachers of non-tested subjects 

59 

88 

Restrictions in rules and regulations on how educators can be compensated 

56 

74 

Concerns or opposition from school staff or staff unions about evaluating 
educators based, at least in part, on student achievement 

50 

80 

Lack of clear state education agency guidance or support on educator 
compensation or evaluation system 

36 

78 

Restrictions in rules and regulations on how educators can be evaluated 

36 

84 

Lack of district staff capacity or expertise to conduct comprehensive educator 
performance evaluations 

26 

85 

Lack of district staff capacity or expertise to develop reliable approaches for 
rating educator performance-based, in part, on student achievement 

22 

78 

Current data systems make linking student test data to individual teachers 
difficult 

19 

88 

Lack of district staff capacity or expertise to identify professional 
development needs of teachers based on performance evaluations 

10 

87 


Table Reads: In 2011-12, 84 percent of the districts rating this challenge perceived insufficient funding to provide differential 
compensation for teachers in high-need areas as a major challenge to implementing educator evaluation and compensation 
systems. Fifty-three percent of districts rated this challenge. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Districts rating the challenge excludes districts that identified the challenge as not applicable and districts that did not 
respond to the question. 

Notes: The percentages in this table are cross-sectional estimates for the population of districts operating in the 2010-11 school 
year. See appendix A for more information about the generalizability of the estimates. Detailed tables in appendix E provide 
confidence intervals for each percentage and the results of significance tests for comparisons of the percentages across 
challenges. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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School-Level Findings 


This section first describes the study's school-level indicators of reform in the areas of educator 
hiring, evaluation, and compensation. We then use the indicators to describe schools' implementation 
of these reforms in 2011-12 and progress from 2009-10 to 2011-12. Next, we compare the percentage 
of low-performing schools in which these reforms were implemented to the corresponding percentages 
among other schools. We conclude this section with a report of the major challenges experienced by 
schools carrying out the workforce development reforms, as reported in 2011-12 by schools. (See 
appendix E for confidence intervals for each percentage reported in this section and the results of 
statistical tests to determine if certain reforms or challenges were more common than others in 2011- 
12.) 

School-Level Reform Indicators 

While most educator workforce policies are made at the state and especially the district levels, 
these policies are not always implemented uniformly within districts. States or districts may pilot these 
reforms in only a subset of schools or target them toward low-performing schools as part of turnaround 
initiatives. In addition, the TIF and SIG programs encouraged districts to implement compensation and 
evaluation reforms in schools serving high-need students (for TIF, defined by the percentage of students 
from low-income families) or in low-performing schools (for SIG, defined in terms of state school 
accountability systems). In order to get a representative picture of the extent to which these reforms 
affected the nation's schools, we examined reform implementation at the school level, where reforms 
will most directly affect students. The four workforce reforms we report on at the school level in this 
chapter are whether: 

• Teachers in the school are being evaluated using multi-level rubrics, multiple observations, and 
student achievement gains; 

• The principal was evaluated based in part on student achievement gains; 

• Compensation for teachers at the school was differentiated based on student achievement 
gains; and 

• Student achievement gains were used to make tenure, dismissal, or reassignment decisions for 
teachers at the school. 

Implementation of Educator Workforce Development Reforms: 2011-12 

• Forty-nine percent of schools reported that principal evaluation was based on student 
achievement gains in 2011-12 (figure 4-6). 

• Thirty-four percent of schools reported that teacher tenure, dismissal, or reassignment 
decisions used student achievement gains in 2011-12. 

• Eighteen percent of the schools reported that teacher evaluation practices included multi- 
level rubrics, multiple observations, and student achievement gains in 2011-12. 
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• Ten percent of schools reported that teacher compensation was differentiated based on 
student achievement gains in 2011-12. 

Progress of Educator Workforce Development Reforms: 2009-10 to 2011-12 

• For only one reform (teacher tenure, dismissal, or reassignment decisions used student 
achievement gains), was there a statistically significant increase in the percentage of schools 
that implemented reforms in 2011-12 than did so in 2009-10 (figure 4-6). 

o Thirty-four percent of schools reported that teacher tenure, dismissal, or reassignment 
decisions used student achievement gains in 2011-12 compared with 29 percent in 2009-10. 
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Figure 4-6. Percentage of schools that used student achievement gains for educator evaluation, 
compensation, and personnel decisions: 2009-10, 2010-11, and 2011-12 
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Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < .05). 

+ Percentage for 2011-12 is significantly different from percentage for 2009-10 (p < .05). 

5 Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 

Figure Reads: Forty-five percent of schools used principal evaluation practices in 2009-10 that included student achievement 
gains. In 2010-11, 50 percent of schools used such evaluation practices, and in 2011-12 49 percent of schools did so. The 
difference between the percentage for 2009-10 and the percentage for 2010-11 is statistically significant. 

Notes: The percentages in this figure are cross-sectional estimates for the population of schools operating in the 2010-11 school 
year. The denominator for these percentages is the estimated number of districts with sufficient data (i.e., answered enough 
questions to calculate the indicator). See appendix A for more information about the generalizability of the estimates. Detailed 
tables in appendix E provide confidence intervals for each percentage and the results of significance tests for comparisons of 
the 2011-12 percentages across indicators. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. 
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Implementation of Educator Workforce Development Reforms by School Performance 
Status: 2011-12 

Improving the performance of students in low-performing schools is a pervasive theme in the 
Recovery Act's education provisions, in particular, RTT, SIG, and TIF. The workforce development 
reforms the act promoted are elements of the federal strategy to improve low-performing schools and 
to promote equitable access to effective educators for students in these schools. For example, educator 
evaluation and teacher compensation as strategies to improve low-performing schools is prominent in 
the SIG transformation school model for low-performing schools. We therefore compared the 
percentage of low-performing schools meeting the reform indicators in 2011-12 to the percentage of 
other schools. 

• In 2011-12, student achievement gains were significantly more likely to be used in low- 
performing schools than in other schools to evaluate principals and to differentiate teacher 
compensation (figure 4-7). 

o Fifty-seven percent of low-performing schools reported that principal evaluation practices 
included student achievement gains compared with 47 percent of other schools. 

o Fourteen percent of low-performing schools reported that teacher compensation was 
differentiated based on student achievement gains compared with 9 percent of other 
schools. 

• In 2011-12, there were no statistical differences between low-performing schools and other 
schools in terms of student achievement gains being used for teacher tenure, dismissal, or 
reassignment decisions or for teacher evaluation, including multi-level rubrics, multiple 
observations, and student achievement gains. 
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Figure 4-7. Comparison of the use of student achievement gains for educator evaluation, 

compensation, and personnel decisions in low-performing schools and in other schools: 

2011-12 
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* Percentage is significantly different from percentage for low-performing schools (p < .05). 

Figure Reads: Fifty-seven percent of low-performing schools reported that their principal evaluation practices included student 
achievement gains in 2011-12. In contrast, 47 percent of all other schools did so. This difference is statistically significant. 

Notes: The percentages in this figure are cross-sectional estimates for the population of schools operating in the 2010-11 school 
year. The denominator is the estimated number of schools in the performance status category (low performing, other) with 
sufficient data (i.e., answered enough questions to calculate the indicator). Low-performing schools include schools that were 
(1) in improvement, corrective action, or restructuring; (2) identified as among the lowest-achieving schools; or (3) have had a 
graduation rate below 60 percent over a number of years. Performance data obtained from approved state applications for 
School Improvement Grants. See appendix A for more information about the generalizability of the estimates. Detailed tables in 
appendix E provide confidence intervals for each percentage. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. Approved state applications for School Improvement Grants 
for school performance status. Retrieved December 2010 from http://www2.ed.gov/programs/sif/ . 
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Challenges Associated With Implementing Educator Workforce Development Reforms: 
2011-12 


Like states and districts, schools also were asked about potential challenges in implementing the 
reforms promoted by the act. The challenges were similar to those we asked of districts, but modified to 
apply to the school context. As with SEAs and districts, a school did not report on a challenge if it did not 
use the relevant reform strategy. 

• Insufficient funding for differentiated compensation was the most frequent major challenge 
reported by schools rating this challenge in 2011-12 (table 4-3). 

o Seventy-three percent of the schools perceived insufficient funding to provide performance- 
based compensation to all eligible teachers as a major challenge. 

o Seventy percent of the schools perceived insufficient funding to provide differential 
compensation for teachers in high-need areas as a major challenge. 

• Challenges related to educator compensation were more likely to be rated as major by schools 
rating these challenges than similar challenges related to educator evaluation in 2011-12. 

o Concerns or opposition from school staff or staff unions about performance-based 

compensation was rated as a major challenge by 55 percent of the schools, while 41 percent 
of schools rated concerns or opposition about evaluating educators based on student 
achievement as a major challenge. 

o Restrictions in rules and regulations regarding how educators can be compensated was 
rated as a major challenge by 47 percent of the schools, while restrictions in rules and 
regulations regarding educator evaluation was rated as a major challenges by 23 percent of 
schools. 
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Table 4-3. Percentage of schools that reported major challenges when implementing educator 
evaluation and compensation systems: 2011-12 


School challenge 

Percent of schools 

Reported challenge 
as a major 
challenge 1 

Rating the 
challenge 2 

Insufficient funding to provide performance-based compensation to 
all eligible teachers 

73 

49 

Insufficient funding to provide differential compensation for teachers 
in high-need areas (e.g., STEM subjects) 

70 

47 

Concerns or opposition from school staff or staff unions about 
performance-based compensation 

55 

64 

Difficulty in measuring student growth for teachers of non-tested 
subjects 

47 

81 

Restrictions in rules and regulations on how educators can be 
compensated 

47 

67 

Concerns or opposition from school staff or staff unions about 
evaluating educators based, at least in part, on student achievement 

41 

74 

Restrictions in rules and regulations on how educators can be 
evaluated 

23 

84 

Lack of school staff capacity or expertise to conduct comprehensive 
educator performance evaluations 

16 

84 

Lack of clear district guidance or support on educator compensation 
or evaluation system 

15 

71 

Limited access to technology needed in order to link student test data 
to individual teachers 

11 

83 

Lack of school staff capacity or expertise to identify professional 
development needs of teachers based on performance evaluations 

8 

86 


Table Reads: In 2011-12, 73 percent of schools rating this challenge perceived insufficient funding to provide performance- 
based compensation to all eligible teachers as a major challenge when implementing educator evaluation and compensation 
systems. Forty-nine percent of schools rated this challenge. 

1 The challenge could be rated as not a challenge, as a minor challenge, or as a major challenge. 

2 Schools rating the challenge excludes schools that identified the challenge as not applicable and schools that did not respond 
to the question. 

Notes: The percentages in this table are cross-sectional estimates for the population of schools operating in the 2010-11 school 
year. See appendix A for more information about the generalizability of the estimates. Detailed tables in appendix E provide 
confidence intervals for each percentage and the results of significance tests for comparisons of the percentages across 
challenges. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 


97 


Comparisons Across Levels 


In this section, we point out similarities and differences among the three levels in the 
implementation of educator workforce reforms and the challenges perceived by those implementing 
them. 

Progress of Educator Workforce Development Reforms: 2009-10 to 2011-12 

Because most of the act's programs and funding affected states or required state-level action, 
we expected that progress in implementing the act's workforce development reforms might differ 
across the three levels of state, district, and school. It may take time for state-level policy changes, 
guidance, or technical assistance to have effects at the district and school levels. At the same time, 
however, some districts and schools may have been experimenting with reforms such as performance- 
based compensation for teachers before their state got involved. We therefore compared the overall 
level of progress among the levels. 

• While the number of SEAs that supported the educator workforce reforms related to principal 
evaluation, teacher evaluation, and teacher compensation increased between 2009-10 and 
2011-12, the percentages of districts and schools implementing these reforms did not increase 
or decreased during this time (figures 4-1, 4-3, and 4-6). 

o The number of SEAs reporting that they supported the use of student achievement gains for 
principal evaluation increased from 6 in 2009-10 to 22 by 2011-12, but the percentages of 
districts and schools doing so did not increase significantly. 

o The number of SEAs reporting that they supported the use of multi-level ratings, multiple 
observations, and student achievement gains for teacher evaluation increased from 1 in 
2009-10 to 14 by 2011-12, but the percentages of districts and schools doing so did not 
significantly increase. 

o The number of SEAs reporting that they supported using student achievement gains to 
differentiate teacher compensation increased from 7 in 2009-10 to 15 by 2011-12; the 
percentage of districts differentiating compensation on this basis decreased significantly, 
and the percentage of schools in which compensation was so differentiated did not 
increase. 
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Challenges Associated With Implementing Educator Workforce Development Reforms: 
2011-12 

Because SEAs, districts, and schools all have somewhat different roles in implementing the 
educator workforce reforms promoted by the Recovery Act, each level of the system is likely to be faced 
with somewhat different challenges. Therefore, the study did not present the same list of potential 
challenges to respondents at each level. There are some challenges, however, that were common, and 
some similarities that are worth noting. 

• Difficulty measuring student growth for teachers of subjects with no standardized tests was 
rated as a major challenge at all three levels in 2011-12. This was the challenge most 
frequently rated as major by SEAs and was the fourth most frequently rated major challenge 
by districts and schools (tables 4-1, 4-2, and 4-3). 

o Thirty-five of the 46 SEAs rating this challenge perceived difficulty in measuring student 
growth for teachers of nontested subjects as a major challenge. 

o Fifty-nine percent of the districts and 47 percent of the schools perceived difficulty in 
measuring student growth for teachers of nontested subjects as a major challenge. 

• At all three levels, concerns or opposition from educators about performance-based 
compensation were more likely to be rated as a major challenge than concerns or opposition 
about evaluation based on student achievement in 2011-12 (tables 4-1, 4-2, and 4-3). 

o Twenty-four of the 34 SEAs rating this challenge, 59 percent of the districts, and 55 percent 
of the schools perceived concerns or opposition from educators or unions about 
performance-based compensation as a major challenge. 

o Twenty-four of the 47 SEAs rating this challenge, 50 percent of the districts, and 41 percent 
of the schools perceived concerns or opposition from educators or unions about evaluating 
educators based, at least in part, on student achievement as a major challenge. 

• Although not asked at the SEA level, large percentages of both districts and schools rated 
insufficient funding to provide performance-based compensation or differential compensation 
as major challenges (tables 4-2 and 4-3). 

o Eighty-three percent of the districts and 73 percent of the schools perceived insufficient 
funding to provide performance-based compensation to all eligible teachers as a major 
challenge. 

o Eighty-four percent of the districts and 70 percent of the schools perceived insufficient 
funding to provide differential compensation for teachers in high-need areas (e.g., STEM 
subjects) as a major challenge. 
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Chapter 5: Improving Low-Performing Schools 


Almost all of the education programs funded by the Recovery Act included a focus on improving 
low-performing schools. These programs offered a multi-pronged approach to improvement that 
included identifying and providing resources to these schools, and increasing the quality of their 
teachers and leaders. Many of the reform strategies promoted by these programs are not new, but built 
on decades of federal efforts to improve the achievement of students in high-poverty and low- 
performing schools. However, the sizable resources of the Recovery Act for turning around low- 
performing schools created new interest in these strategies and funded some new requirements. 

Since the initial authorization of ESEA, the Title I, Part A program has provided funding to 
improve the educational opportunities and achievement of students in schools with the highest 
percentages or numbers of children from low-income families within each LEA. With the subsequent 
reauthorizations of ESEA in 1994 and 2001, federal policy evolved toward identifying and providing 
resources to low-performing schools through school accountability provisions. ESEA required SEAs and 
LEAs to identify Title I schools consistently not meeting achievement targets and, in turn, required those 
schools to develop improvement plans. ESEA also required LEAs to offer students at these schools the 
option of transfer to another public school in the district, including a charter school. More prescriptive 
requirements (e.g., extending instructional time, reopening as a public charter school, or replacing all or 
most of the staff) apply as the length of time that the school fails to meet achievement targets 
increases. The 2001 ESEA reauthorization (NCLB) also authorized the School Improvement Grant (SIG) 
program through which SEAs make sub grants to LEAs to help their low-performing schools comply with 
school improvement, restructuring, and corrective action requirements. SIG received its first 
appropriations in fiscal year 2007. 

Other federal efforts to improve student achievement focused on improving the quality of 
teachers and encouraging the placement of highly qualified or effective teachers in high-need schools. 
NCLB included the requirement that all core academic subject teachers be highly qualified and, under 
Title II, Part A, provides funds to prepare, recruit, and retain highly qualified teachers in high-need 
schools. To help increase the pool of highly effective educators serving high-need schools, the TIF 
program, established in 2006, encourages districts to develop and implement in high-need schools 
compensation systems tied to teacher and principal performance. 

The Recovery Act continued the federal policy of identifying and providing resources to low- 
performing schools, 55 but also brought attention, substantial resources, and intensive reform 
requirements to the PLA schools. These are the state's lowest-achieving 5 percent of schools and 
secondary schools with chronically low graduation rates. 56 The act provided an additional $3 billion for 


For example, the act provided $10 billion for Title I, Part A allowing the program to serve more students and improve the 
quality of services provided by implementing evidenced-based strategies to build capacity for improving teaching and 
learning in Title I schools (U.S. Department of Education, 2010c). 

PLA schools means, as determined by the state: (a) Any Title I school in improvement, corrective action, or restructuring that 
— (i) Is among the lowest-achieving five percent of Title I schools in improvement, corrective action, or restructuring or the 
lowest-achieving five Title I schools in improvement, corrective action, or restructuring in the State, whichever number of 
schools is greater; or (ii) Is a high school that has had a graduation rate that is less than 60 percent over a number of years; 
and (b) Any secondary school that is eligible for, but does not receive. Title I funds that — (i) Is among the lowest-achieving 
five percent of secondary schools or the lowest-achieving five secondary schools in the State that are eligible for, but do not 
receive, Title I funds, whichever number of schools is greater; or (ii) Is a high school that has had a graduation rate that is less 
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the SIG program. This large infusion of funds led to new program requirements. In general, SEAs are 
required to (1) identify PLA schools, (2) competitively award SIG funds to districts that commit to 
implementing specific intervention models, and (3) provide technical assistance to LEAs and 
participating schools. The required intervention models— the transformation, restart, closure, and 
turnaround models— focus on significant changes to a school such as leadership and staff changes, 
closure, or increased student learning time. These models required a more comprehensive approach to 
reform than the menu of strategies offered for schools in corrective action and restructuring under 
NCLB. 57 LEAs have the critical role of implementing the models and monitoring the progress of each SIG 
school. The SFSF and RTT programs also emphasized the focus on PLA schools, with roles for states to 
report on PLA schools or support the LEAs that are implementing the school intervention models. 

Building upon previous efforts to ensure that qualified teachers teach students in high-need 
schools, the Recovery Act programs encouraged the equitable distribution of effective teachers and 
principals in schools. The RTT selection criteria considered state and district plans to ensure that 
students in high-need schools are taught by effective educators. SFSF required SEAs to report on the 
distribution of effective teachers. The act also increased funding for TIF, which enabled an increase in 
the number of grants for districts to develop compensation systems that could entice highly effective 
educators to serve in low-performing schools. 

The Recovery Act also promoted a more favorable environment for public charter schools, 
potentially increasing the pool available for public school choice. Public charter schools were options for 
public school choice and restructuring provisions for low-performing schools under NCLB, and similar 
provisions exist under the SIG restart and school closure intervention models. However, through the 
SFSF and RTT programs, states were encouraged to remove prohibitions on the existence of charter 
schools and their numbers and to promote accountability for these schools. 

As in the act's other assurance areas, fully realizing the act's intent to achieve major 
improvements in low-performing schools entails actions at state, district, and school levels. Each level 
has a somewhat different role, and the act's components varied in their emphasis on state, district, and 
school actions. To provide a picture of the prevalence and progress of the reforms promoted by the Act, 
we identified and examined indicators of reform at all three levels. 

Using these indicators, this chapter describes the prevalence of reforms at all three levels in 
2011-12, and, for SEAs and schools, progress from 2009-10 to 2011-12. 58 We also report on major 
challenges to implementing these reforms as perceived by survey respondents at each level. Our review 
of state implementation includes all states and their SEAs. At the district level, our reporting is limited to 
districts with at least one low-performing school in 2011-12. Similarly, at the school level, our reporting 
is limited to schools identified as low performing. 59 As done in previous chapters, this chapter also 


than 60 percent over a number of years. (U.S. Department of Education, November 2010, Guidance on FY2010 School 
Improvement Grants, p. 1. downloaded from: http://www2.ed.gov/programs/sif/sigguidancell012010.pdf ). 

57 For example, under NCLB, LEAs were required to take at least one of a set of corrective actions or restructuring arrangements 
for schools in corrective action or restructuring, respectively. The SIG intervention models required that schools undergo a 
series of reform actions. 

58 The analysis of districts with low-performing schools was limited to the 2011-12 school year for two reasons. First, districts 
were asked to report whether they had low-performing schools in 2010-11 and 2011-12 but were not asked about this status 
for 2009-10. Second, the size of the confidence intervals for analyses of 2010-11 data raised questions about the accuracy of 
the estimates. 

59 For this report, low-performing schools were identified through state FY 2010 SIG applications and include: (1) any ESEA Title 
l-eligible school designated for improvement, corrective action, or restructuring and (2) any high school, regardless of Title I 
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compares prevalence of reform during the final year of the study for those states that were awarded 
RTT grants and other states. However, district comparisons by district size and poverty level were not 
done because the size of the confidence intervals for these analyses of 2011-12 data raised questions 
about the accuracy of the estimates. Also, given the focus of the chapter, the school analyses are limited 
to low-performing schools. The chapter ends by comparing progress and challenges across levels. 


Key Findings Across Levels 

• While the number of SEAs providing guidance on choosing and 
implementing school intervention models increased during the period to 
include all 51 SEAs in 2011-12, reports from low-performing schools and 
their districts suggest that relatively little use was made of key elements 
of the SIG closure, restart, and turnaround models (figures 5.1, 5.3, and 
5.4a). 

• Across levels, the implementation of compensation incentives to improve 
staffing at low-performing schools remained low during the study period 
(figures 5.1, 5.3, and 5.4a). 

• Insufficient funding to make substantial changes to the school day or year 
was the second most frequently rated major challenge for districts and 
schools in 2011-12. At the state level, restrictions in rules and regulations 
regarding extension of the school day or year were not rated among the 
top two major challenges by SEAs in 2011-12 (tables 5.1, 5.2, and 5.3). 

• Relatively few SEAs, districts, or schools reported lack of guidance or 
support from the governmental level on implementing school reform 
models as a major challenge in 2011-12. 


State-Level Findings 


In this section, we first describe the study's state-level indicators of reform to improve low- 
performing schools. We then use the indicators to describe SEAs' implementation of these reforms in 
2011-12 and progress from 2009-10 to 2011-12. Next, we report on whether a greater percentage of 
RTT states implemented these reforms in 2011-12 compared with states that did not receive RTT grants. 
Last, we conclude this section with a report of the major challenges related to the improvement of low- 
performing schools, as reported in 2011-12 by SEAs. 


funding or status, with a cohort graduation rate (i.e., percent of ninth graders who graduate within 4 or 5 years) less than 60 
percent over the last several years. 
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State-Level Reform Indicators 

Through Recovery Act programs, states have the responsibility to identify the lowest-performing 
schools across the state and create favorable conditions for charter schools. The programs also 
encouraged states to support districts as they direct implementation school intervention models by 
providing technical assistance and developing collaborative plans. As a result, we included four 
indicators at the state level. These are whether the SEA: 

• Allowed expansion of the number of charter schools in the state; 

• Provided guidance on choosing and implementing the school intervention models defined by 
ED; 

• Monitored deployment of effective educators in low-performing schools; and 

• Supported the use of compensation incentives to improve staffing in low-performing schools. 

We included the expansion of charter school indicator because the RTT selection criteria gave 
preference to states that did not prohibit or inhibit an increase in the number of charter schools. In 
addition, SFSF reporting requirements for states included the number of charter schools permitted to 
operate under state law and the number of charter schools currently operating in the state. 

The RTT selection criteria also included whether the state had a high-quality plan and ambitious 
targets to support districts in turning around low-performing schools by implementing one of the four 
SIG school intervention models. We included the indicator about guidance on choosing and 
implementing school intervention models because SEA assistance and guidance in this area is important 
as districts may have little experience in selecting such models. 

The inclusion of our third SEA indicator, monitoring deployment of effective educators in low- 
performing schools, flows from the RTT selection criteria about whether the state, and its districts, had a 
plan to ensure the equitable distribution of these educators in high-poverty and high-minority schools. 
The plan also is to ensure that students in these schools are taught by effective educators. SFSF also 
required SEAs to report on the distribution of effective teachers. RTT state plans to promote the 
equitable distribution of effective educators could include the use of compensation incentives, 
therefore, we also included an indicator of states' support for such incentives to improve staffing in low- 
performing schools. See appendix B for the components, decision rules, and specific Recovery Act 
requirements embodied in each indicator. 

Implementation of Reforms to Support Improvement in Low-Performing Schools: 
2011-12 

• All 51 SEAs reported in 2011-12 that they provided guidance to districts on choosing and 
implementing the school intervention models defined by ED (figure 5.1). 

• Thirty-three states' policies encouraged expansion in the number of charter schools in 2011-12 
according to the National Alliance of Public Charter schools. 
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• Less than 15 SEAs reported in 2011-12 that they supported specific reforms to improve the 
quality of educators in low-performing schools including monitoring the deployment of 
effective educators in low-performing schools. 

o Fourteen SEAs reported that they supported compensation incentives to improve staffing in 
low-performing schools. 

o Twelve SEAs reported that they monitored deployment of effective educators in low- 
performing schools. 

Progress of Reforms to Support Improvement in Low-Performing Schools: 2009-10 to 
2011-12 

• The number of SEAs reporting support for the school intervention models increased from 
2009-10 to 2011-12, with nine more SEAs providing guidance and choosing and implementing 
the models by 2011-12 (figure 5-1). 

• Fourteen more states allowed for expansion of the number of charter schools in 2011-12 than 
in 2009-10 (increased from 19 to 33 SEAs). 

• The number of SEAs implementing reforms focused on improving educator quality increased 
from 2009-10 to 2011-12. 

• Six more SEAs supported the use of compensation incentives to improve staffing at low- 
performing schools (increased from 8 to 14 SEAs). 

• Five more SEAs monitored deployment of effective educators in low-performing schools 
(increased from 7 to 12 SEAs). 
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Figure 5-1. Number of state education agencies (SEAs) that implemented reforms to support 
improvement in low-performing schools: 2009-10, 2010-11, and 2011-12 
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Figure Reads: Forty-two SEAs reported that in 2009-10 they provided guidance on choosing and implementing the school 
intervention models defined by ED. Fifty SEAs did so in 2010-11, and all SEAs did so in 2011-12. 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in a previous year, the state is 
counted as meeting the indicator in subsequent years. See appendix A for more information about how data for previous 
years were used in these indicators. 

2 In measuring state reform activity for this indicator, if in a previous year a state passed legislation to either increase the 
permissible number of charter schools or remove prohibitions on charter schools, the state is counted as meeting the 
indicator in subsequent years. 

3 To meet this indicator, SEAs not only had to play a role in supporting implementation, but also had to be supporting the 
specific types of systems that the Recovery Act explicitly identified. Two SEAs that met the indicator in 2010-11 are counted 
as meeting the indicator in 2011-12 given that their role in supporting implementation was a potentially ongoing or continued 
activity. See appendix A for more information about how data for previous years were used in these indicators. 

Note: Respondents include 50 states and DC. 

Sources: National Alliance for Public Charter Schools report: Measuring up to the model: A ranking of state charter school laws 
(2010 and 2011) for information on the expansion of charter schools and U.S. Department of Education, Institute of Education 
Sciences, Charting the Progress of Education Reform: An Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 
State Education Agency Surveys for information on implementation of the school intervention models for low-performing 
schools, compensation incentives, and the movement and deployment of effective educators. 
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Implementation of Reforms to Support Improvement in Low-Performing Schools by 
State RTT Status: 2011-12 

Recognizing the emphasis in the RTT program on state-level action to improve low-performing 
schools, we expected that by 2011-12, a higher percentage of states that had won RTT grants would 
have adopted and implemented the Recovery Act reforms focused on low-performing schools with 
other states. In addition, the resources available through the RTT grants could have helped to finance 
these SEAs' provision of supports for improving low-performing schools. 

• In 2011-12, states that received RTT grants were more likely than other states to have 
adopted policies that encouraged expansion in the number of charter schools. Eleven of the 
12 RTT states adopted policies that encouraged the expansion in the number of charter schools 
compared with 22 of the 39 other states (figure 5-2). 

• In 2011-12, SEAs in RTT states were more likely than other states to report that they 
supported reforms aimed at improving the quality of educators in low-performing schools. 

o Seven of the 12 SEAs in RTT states reported that they supported the use of compensation 
incentives to improve staffing at low-performing schools compared with 7 of 39 SEAs in 
other states. 

o Five of the 12SEAs in RTT states reported that they monitored the deployment of effective 
educators in low-performing schools compared with 7 of the 39 SEAs in other states. 
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Figure 5-2. Comparison of the implementation of reforms to support improvement in low- 
performing schools in Race to the Top (RTT) states and in other states: 2011-12 
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Figure Reads: All 12 state education agencies (SEAs) in RTT states and all 39 SEAs in other states reported that in 2011-12 they 
provided guidance on choosing and implementing the school intervention models defined by ED (100 percent). 

1 In measuring state reform activity for this indicator, if an SEA reported that it took action in a previous year, the state is 
counted as meeting the indicator in subsequent years. See appendix A for more information about how data for previous 
years were used in these indicators. 

2 In measuring state reform activity for this indicator, if in a previous year a state passed legislation to either increase the 
permissible number of charter schools or remove prohibitions on charter schools, the state is counted as meeting the 
indicator in subsequent years. 

3 To meet this indicator, SEAs not only had to play a role in supporting implementation, but also had to be supporting the 
specific types of systems that the Recovery Act explicitly identified. Two SEAs that met the indicator in 2010-11 are counted 
as meeting the indicator in 2011-12 given that their role in supporting implementation was a potentially ongoing or continued 
activity. See appendix A for more information about how data for previous years were used in these indicators. 

Notes: Respondents include 50 states and DC. RTT states are the 12 states awarded RTT grants in the first two rounds of 
competition (DE, DC, FL, GA, HI, MA, MD, NC, NY, OH, Rl, TN). 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey. 
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Challenges Associated With SEA Reforms to Support Improvement in Low-Performing 
Schools: 2011-12 

SEAs that had adopted reforms to support low-performing schools were asked to report on the 
challenges associated with these reform efforts. An SEA did not report on challenges related to a 
particular reform strategy if it was not implementing that strategy. 

• The three of the four most frequently reported major challenges in 2011-12 concerned 
restrictions in rules and regulations affecting reform efforts (table 5-1). 

o Fifteen of the 49 SEAs rating this challenge rated restrictions in rules and regulations 
regarding the extent of autonomy that districts and schools can be granted in terms of 
staffing or budgets as a major challenge. 

o Thirteen of the 45 SEAs rating this challenge rated restrictions in rules and regulations 
regarding teacher hiring practices as a major challenge. 

o Twelve of the 48 SEAs rating this challenge rated restrictions in rules and regulations 
regarding extension of the school day or year as a major challenge. 

• Concerns or opposition from educators about school closing or restructuring schools was rated 
as a major challenge by 13 of the 47 SEAs rating this challenge. 

• Lack of SEA staff or expertise to help guide or develop district staff on reforms related to low- 
performing schools were rated as a major challenge by five or fewer SEAs rating these 
challenges. The few number of SEAs reporting lack of staff or expertise is notable because of 
findings from the National Assessment of Title I schools that reported that "most states (42) 
reported that providing assistance to all schools identified for improvement was a moderate 
or serious challenge in 2003-04"(lnstitute of Education Sciences, (2007), p. xxviii) 
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Table 5-1. Number of state education agencies (SEAs) that reported major challenges in efforts to 
support improvement in low-performing schools: 2011-12 


SEA challenge 

Number of SEAs 
that reported 
challenge as a major 
challenge 1 

Total number 
of applicable 
SEAs 2 

Restrictions in rules and regulations regarding extent of autonomy that 
local education agencies and schools can be granted in terms of staffing 
or budgets 

15 

49 

Restrictions in rules and regulations regarding teacher hiring practices 

13 

45 

Concerns or opposition from educators about closing or restructuring 
schools 

13 

47 

Restrictions in rules and regulations regarding extension of school day 
or year 

12 

48 

Restrictions in rules and regulations regarding number of schools that 
can be closed, opened as charters, or restructured in other ways 

4 

38 

Lack of SEA staff or expertise to provide guidance and technical 
assistance on whole-school reform or turnaround models to LEAs 

5 

51 

Lack of SEA staff or expertise to provide professional development 
focused on improving low-performing schools 

4 

50 

Lack of clear federal guidance or support focused on implementing 
whole-school reform or turnaround models 

3 

49 

Lack of SEA staff or expertise to identify and disseminate best practices 
concerning improving low-performing schools 

3 

50 

Lack of SEA staff or expertise to screen and disseminate information on 
EMOs, CMOs and school turnaround experts 3 

1 

32 


Table Reads: In 2011-12, 15 of the 49 SEAs rating this challenge perceived restrictions in rules and regulations regarding extent 
of autonomy that LEAs and schools can be granted in terms of staffing or budgets as a major challenge to supporting the 
improvement of low-performing schools. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Applicable SEAs are those that rated the challenge. This excludes SEAs that identified the challenge as not applicable and SEAs 
that did not respond to the question. 

3 An EMO is an education management organization. A CMO is a charter management organization. 

Note: Respondents include 50 states and DC. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey. 
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District-Level Findings 


This section begins by describing the study's district-level indicators of reform to improve low- 
performing schools. We then use the indicators to describe districts' implementation of these reforms in 
2011-12. 60 The analysis is limited to districts with at least one low-performing school. Last, we conclude 
this section with a report of the major challenges in supporting school improvement, as reported in 
2011-12 by districts. (See appendix F for confidence intervals for each percentage reported in this 
section and the results of statistical tests to determine if certain reforms or challenges were more 
common than others in 2011-12.) 

District- Level Reform Indicators 

Low-performing schools are common in American districts, with 45 percent of all districts 
reporting in 2011-12 that one or more of their schools was low performing. 61 Districts play the central 
role in directing and monitoring the implementation of reforms intended for achieving significant 
improvement in low-performing schools, as emphasized under the act. At the district level, we included 
nine indicators of reform in areas over which the district generally has primary jurisdiction. They 
represent the major schooling changes promoted to improve student achievement in low-performing 
schools. These indicators are whether the district: 

• Replaced principals and teachers in low-performing schools; 

• Contracted with an external organization to operate low-performing schools; 

• Provided compensation incentives to improve staffing at low-performing schools; and 

• Provided technical assistance to low-performing schools to screen or select school improvement 
experts or models. 

• Targeted low-performing schools for closure; 

• Required low-performing schools to partner with organizations that specialize in instructional 
improvement; 

• Provided school leaders in low-performing schools with staffing or budgeting flexibility to 
implement school reform; 

• Extended the school day, week, or year in low-performing schools; and 


60 The analysis of districts with low-performing schools was limited to the 2011-12 school year, for two reasons. First, districts 
were asked to report whether they had low-performing schools in 2010-11 and 2011-12 but were not asked about this status 
for 2009-10. Second, the size of the confidence intervals for analyses of 2010-11 data raised questions about the accuracy of 
the estimates. 

61 For this report, low-performing schools include: (1) any ESEA Title l-eligible school designated for improvement, corrective 
action, or restructuring and (2) any high school, regardless of Title I funding or status, with a cohort graduation rate (i.e., 
percent of ninth graders who graduate within 4 or 5 years) less than 60 percent over the last several years. 
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• Implemented programs in low-performing schools to encourage family and community 
involvement. 

All nine of these indicators are components of the four SIG school intervention models included 
in the SIG program regulations that stemmed from the increase in funds from the Recovery Act. They 
are as follows: 

• Turnaround model, which entails replacement of the school principal; screening and rehiring no 
more than 50 percent of existing staff; providing appropriate social-emotional and community- 
oriented services and supports for students; and installation of new procedures to improve staff 
competencies, school governance, curricula, data use, and accountability and to increase 
learning time; 

• Restart model, which entails conversion of a school or LEA to charter school status or to 
operation by an education management organization; 

• School closure, which entails closing the low-performing school and enrolling the school's 
students in other, higher performing schools; and 

• Transformation model, which includes many of the methods outlined for the turnaround model 
but also includes options for the school to offer additional compensation to staff, measure 
changes in instructional practices, and ensure that the school is not required to accept a teacher 
without the mutual consent of the teacher and principal (regardless of the teacher's seniority). 
This model also allows districts to use SIG resources to create community-oriented schools. In 
addition, the model requires districts to deliver (or facilitate the delivery of) ongoing, intensive 
technical assistance and related support from the district, the SEA, or an external partner 
organization, such as a school turnaround organization or an education management 
organization. 

Implementation of Reforms to Support Improvement in Low-Performing Schools: 
2011-12 

• Among districts with one or more low-performing schools, the most common support for low- 
performing schools in 2011-12 reported by 78 percent of districts was implementing programs 
that encouraged family and community involvement (figure 5-3; see appendix table F-3 for 
results of statistical comparisons of differences across indicators). 

• The next most common group of reforms in 2011-12 reported by 40 to 50 percent of these 
districts included providing technical assistance to low-performing schools to screen or select 
school-improvement experts or models; extending the school day, week, or year; and 
requiring low-performing schools to partner with external organizations (figure 5-3). 

o Fifty percent of these districts provided technical assistance to low-performing schools to 
screen or select school improvement experts or models. 
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o Forty percent of these districts reported that they extended the school day, week, or year in 
low-performing schools. 

o Thirty-nine percent reported that they required low-performing schools to partner with 
organizations that specialize in instructional improvement. 

• Less than a quarter of districts with low-performing schools reported in 2011-12 that they 

implemented the other reforms the study tracked. 

o Twenty-three percent of districts with low-performing schools reported that they provided 
school leaders in these schools with staffing or budgeting flexibility to implement school 
reform. 

o Sixteen percent of these districts reported that they provided compensation incentives to 
improve staffing at low-performing schools. 

o Five percent of districts with one or more low-performing schools reported in 2011-12 that 
district policies or programs called for replacing both the principal and a substantial 
proportion of teachers in the district's low-performing schools. 

■ However, when examining policies or programs to replace principals and 
teachers separately, an additional 11 percent of districts with low-performing 
schools had adopted policies or programs to replace principals in individual low- 
performing schools but had not adopted corresponding policies or programs for 
replacing a substantial proportion of teachers (appendix table F-l). 

o Small percentages of districts reported that they contracted with an external organization to 
operate low-preforming schools (3 percent) or targeted low-performing schools for closure 
(2 percent). 
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Figure 5-3. Percentage of districts with low-performing schools that implemented reforms to support 
improvement in low-performing schools: 2011-12 
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Figure Reads: Seventy-eight percent of districts with low-performing schools reported that in 2011-12 they implemented 
programs to encourage family and community involvement. 

Notes: The percentages in the figure are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year and have low-performing schools. Low-performing schools include: (1) any ESEA Title l-eligible school 
designated for improvement, corrective action, or restructuring and (2) any high school, regardless of Title I funding or status, 
with a cohort graduation rate (i.e., percent of ninth graders who graduate within 4 or 5 years) less than 60 percent over the last 
several years. See appendix A for more information about the generalizability of the estimates. In this figure, the denominator 
is the estimated number of districts with sufficient data (i.e., answered enough questions to calculate the indicator) that 
reported in the evaluation's 2012 district survey they had low-performing schools in that year. Detailed tables in appendix F 
provide confidence intervals for each percentage and the results of significance tests for comparisons of the 2011-12 
percentages across indicators. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Challenges for Districts in Supporting School Improvement: 2011-12 

Districts with low-performing schools reported on the major challenges that they experienced in 
supporting school improvement. In implementing strategies to improve low-performing schools, 
districts may decide to redesign school operations and personnel policies, often by opening doors to 
new schools and new sources of schooling expertise. Yet, even when the community agrees that 
improvements are needed, such changes can sometimes be unpopular and difficult to achieve. Some of 
the reforms, such as extending the school day or providing school improvement experts, can also 
require additional funds. As with SEAs, a district did not report on challenges related to a particular 
reform strategy if it was not implementing that strategy. 

• Among districts with one or more low-performing schools, insufficient funding for a reform 
effort was reported most frequently as a major challenge among districts rating these 
challenges in 2011-12 (table 5-2). 

o Sixty-five percent reported that insufficient funding to implement whole-school or 
turnaround intervention models was a major challenge. 

o Fifty-seven percent reported that insufficient funding to make substantial changes to school 
day or year schedules was a major challenge. 

• The quality of data systems and lack of evidence about the effectiveness of school models 
were reported least frequently as a major challenge among districts rating these challenges in 
2011 - 12 . 

o Ten percent of districts reported that difficulty in tracking the success of school 
improvement efforts with current data systems was a major challenge. 

o Nine percent of districts reported that a lack of evidence about effectiveness of school 
improvement models was a major challenge. 
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Table 5-2. Percentage of districts with low-performing schools that reported major challenges when 
supporting school improvement: 2011-12 


District challenge 

Percent of districts 

Reported challenge 
as a major challenge 1 

Rating the challenge 2 

Insufficient funding to implement whole-school or turnaround 
intervention models 

65 

65 

Insufficient funding to make substantial changes to school day or 
year schedules 

57 

80 

Restrictions in rules and regulations regarding extent of 
autonomy that schools can be granted in terms of staffing or 
budgets 

48 

68 

Insufficient funding to support school-based experts (outside 
consultants, instructional specialists or coaches, mentors) 

45 

93 

Insufficient funding to support special programs for students and 
families 

41 

92 

Concerns or opposition from parents or community groups about 
closing or restructuring schools 

41 

50 

Restrictions in rules and regulations regarding extension of 
school days or years 

40 

68 

Insufficient help from local social services and other community- 
based organizations in providing services to students and their 
families 

38 

87 

Restrictions in rules and regulations regarding number of schools 
that can be closed, opened as charters, or restructured in other 
ways 

33 

27 

Lack of district staff capacity or expertise to screen or provide 
guidance or advice about EMOs and CMOs 3 

31 

32 

Unwillingness of high-performing teachers to move to low- 
performing schools 

30 

38 

Lack of district staff capacity or expertise to provide guidance or 
advice concerning whole-school or turnaround intervention 
models 

29 

66 

Lack of evidence about performance of CMOs or EMOs or other 
intervention experts 3 

22 

32 


continued 
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Table 5-2. Percentage of districts with low-performing schools that reported major challenges when 
supporting school restructuring and improvement: 2011-12 (cont'd) 


District challenge 

Percent of districts 

Reported challenge as 
a major challenge 1 

Rating the challenge 2 

Lack of clear state education agency guidance or support 
focused on adoption of whole-school reform models 

15 

54 

Lack of district staff capacity or expertise to train instructional 
specialists, coaches, lead teachers, or school-based 
professional development staff 

15 

94 

Current data systems make tracking the success of school 
improvement efforts difficult 

10 

96 

Lack of evidence about effectiveness of school improvement 
models 

9 

77 


Table Reads: In 2011-12, 65 percent of the districts with low-performing schools rating this challenge perceived insufficient 
funding to implement whole-school or turnaround intervention models as a major challenge to supporting school restructuring 
and improvement. Sixty-five percent of district with low-performing schools rated this challenge. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Districts rating the challenge excludes districts that identified the challenge as not applicable, districts that did not respond to 
the question, and districts without low-performing schools. 

3 An EMO is an education management organization. A CMO is a charter management organization. 

Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year and have low-performing schools. Low-performing schools include: (1) any ESEA Title l-eligible school 
designated for improvement, corrective action, or restructuring and (2) any high school, regardless of Title I funding or status, 
with a cohort graduation rate (i.e., percent of ninth graders who graduate within 4 or 5 years) less than 60 percent over the last 
several years. See appendix A for more information about the generalizability of the estimates. Detailed tables in appendix F 
provide confidence intervals for each percentage. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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School-Level Findings 


This section first describes the study's school-level indicators of reform to stimulate 
improvement in low-performing schools. We then use the indicators to describe low-performing 
schools' implementation of these reforms in 2011-12 and progress from 2009-10 to 2011-12. Last, we 
conclude this section with a report of major challenges reported by low-performing schools in 2011-12 
as they worked to implement these reforms. (See appendix F for confidence intervals for each 
percentage reported in this section and the results of statistical tests to determine if certain reforms or 
challenges were more common than others in 2011-12.) 

School-Level Reform Indicators 

In 2011-12, 18 percent of the nation's public schools were identified as low performing. 62 In the 
effort to achieve improvements in low-performing schools, the schools themselves have distinct 
opportunities and responsibilities to carry out the improvement strategies and models. The study 
examined 11 school-level indicators of reform. We examined so many indicators because so many 
improvement strategies have been advocated for and applied to low-performing schools. Nine of the 11 
indicators cover practices required or permitted in one or more of the SIG school intervention models. 
One indicator, reassigning effective teachers as part of restructuring, was proposed by ED as a remedy 
for unequal access to effective educators, most recently in the Administration's 2010 blueprint for re- 
authorizing the ESEA (U.S. Department of Education, 2010b). The other indicator, using outside school 
improvement experts, is a strategy that has been recommended and used for many years (e.g., 
Stringfield, Ross, and Smith, 1996). 

Three indicators focus on parents, families, and communities and on students' social-emotional 
development. These include whether the school implemented programs to: 

• Encourage family and community involvement; 

• Address students' social and emotional needs; and 

• Orient parents to school improvement models. 

Four indicators focus on school scheduling or organization. These include whether the school: 

• Modified the daily schedule to increase instructional time in reading/English language arts or 
mathematics; 

• Used outside school improvement experts as part of school restructuring or to improve 
instruction; 


' For this report, low-performing schools include schools that were (1) in improvement, corrective action, or restructuring, 

(2) identified as among the lowest-achieving schools, or (3) have had a graduation rate below 60 percent over a number of 
years. Performance data obtained from approved state applications for FY 2010 SIG. See the methodology appendix for more 
information. 
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Extended the school day, week, or year as part of school restructuring; and 




• Received increased staffing or budgetary autonomy as part of school restructuring. 

Implementation of Reforms to Support Improvement in Low-Performing Schools: 
2011-12 

• Reforms involving programs for families or communities and addressing the social and 
emotional needs of students were the most common, with over three-quarters of low- 
performing schools in 2011-12 reporting implementation (figure 5-4). 

o Ninety-eight percent of these schools reported that they implemented programs to 
encourage family and community involvement. 

o Ninety-one percent reported that they implemented programs to address students' social 
and emotional needs. 

o Seventy-nine percent reported that they implemented programs to orient parents to school 
improvement models. 


• Of the reforms relating to scheduling and organization, increasing instructional time and using 
outside school improvement experts were the most commonly reported in 2011-12 by low- 
performing schools. 

o Modifying the daily schedule to increase instructional time for reading/English language arts 
or mathematics (78 percent) and the use of outside school improvement experts as part of 
school restructuring or to improve instruction (66 percent) were reported most frequently. 

o Less frequently reported were extending the school day, week, or year (19 percent) or 
increasing school autonomy in staffing or budgeting (9 percent) as part of restructuring 
efforts. 

• Small percentages of low-performing schools (5 to 21 percent) reported in 2011-12 that they 
implemented reforms to improve the teacher workforce through nonfinancial incentives, 
teacher replacement, compensation incentives, or reassignment. 

o Twenty-one percent of low-performing schools reported that nonfinancial incentives were 
used to encourage teachers to move to or remain in the school. 

o Eleven percent of low-performing schools reported that a substantial portion of teachers 
were replaced as part of school restructuring. 

o The use of compensation incentives to encourage teachers who move to teach at the school 
and reassignment of effective teachers to the school as part of school restructuring were 
reported by 5 percent of low-performing schools. 
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Figure 5-4. Percentage of low-performing schools that supported improvement: 2009-10, 2010-11, 
and 2011-12 


School implemented programs to encourage family and 
community involvement 

School implemented programs to address students' 
social and emotional needs 

School implemented programs to orient parents to 
school improvement models 

School modified daily schedule to increase instructional 
time for reading/English language arts or mathematics 

School used outside school improvement experts as part 
of school restructuring or to improve instruction 

Nonfinancial incentives to encourage teachers to move 
to or remain in the school 

School day, week, or year extended as part of school 
restructuring 
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Percent of low-performing schools 

Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < .05). 
tPercentage for 2011-12 is significantly different from percentage for 2009-10 (p < .05). 

§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < .05). 

Figure Reads: Eighty-eight percent of low-performing school reported that in 2009-10 they implemented programs to 
encourage family and community involvement. Ninety-five percent of low-performing schools implemented such programs. in 
2010-11, and 98 percent did so in 2011-12. 

Notes: The percentages in the figure are cross-sectional estimates for the population of low-performing schools that were 
operating in the 2010-11 school year. Low-performing schools include schools that were (1) in improvement, corrective action, 
or restructuring; (2) identified as among the lowest-achieving schools; or (3) have had a graduation rate below 60 percent over 
a number of years. See appendix A for more information about the generalizability of the estimates. In this figure, the 
denominator is the estimated number of low-performing schools with sufficient data (i.e., answered enough questions to 
calculate the indicator). Detailed tables in appendix F provide confidence intervals for each percentage and the results of 
significance tests for comparisons of the 2011-12 percentages across indicators. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. Approved state applications for School 
Improvement Grants for low-performing schools data. Retrieved December 2010 from http://www2.ed.gov/programs/sif/ . 
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Progress of Reforms to Support Improvement in Low-Performing Schools: 2009-10 to 
2011-12 


• The percentage of low-performing schools that implemented programs to increase family and 
community involvement and parent awareness of school improvement models increased 
significantly over the 3-year period (figure 5-4). 

o A greater percentage of low-performing schools reported that they implemented programs 
to encourage family and community involvement in 2011-12 (98 percent), compared with 
2009-10 (88 percent). 

o A greater percentage reported that they implemented programs to orient parents to school 
improvement models in 2011-12 (79 percent), compared with 2009-10 (69 percent). 

o A greater percentage of low-performing schools reported in 2011-12 that they implemented 
programs to address students' social and emotional needs than did so in 2009-10 (91 
percent, compared with 81 percent). 

• Reforms relating to scheduling and organization were reported by significantly greater 
percentages of low-performing schools in 2011-12 than in 2009-10. 

o The percentage of low-performing schools that reported that they modified the daily 
schedule to increase instructional time for reading/English language arts or mathematics 
increased from 58 percent in 2009-10 to 78 percent in 2011-12. 

o The percentage of low-performing schools reporting that they used school improvement 
experts as part of school restructuring or to improve instruction increased from 47 percent 
in 2009-10 to 66 percent in 2011-12. 

o The percentage that reported that they extended the school day, week, or year as part of 
school restructuring increased from 12 percent in 2009-10 to 19 percent in 2011-12. 

o The percentage of low-performing schools reporting that staffing or budgeting autonomy 
increased as part of school restructuring increased significantly from 5 percent in 2009-10 to 
9 percent in 2011-12. 

• The percentage of low-performing schools implementing reforms affecting the teacher 
workforce in order to support school improvement did not change significantly from 2009-10 
to 2011-12. 
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Challenges for Low-Performing Schools When Working on School Organization and 
Improvement: 2011-12 

Low-performing schools reported on the challenges experienced in implementing the reforms 
promoted by the act in the areas of school organization and improvement. Schools potentially faced 
many of the same challenges as districts, such as restrictions on or lack of funding for, structural or 
programmatic reforms, or opposition from the community. As with SEAs and districts, a school did not 
report on a challenge if it did not use the relevant reform strategy. 

• The major challenge cited by the largest percentage of low-performing schools was 
restrictions in rules and regulations on replacing less effective teachers, as reported by 49 
percent of the low-performing schools rating this challenge (table 5-3). 

• Insufficient funding to make substantial changes to school day or year schedules was reported 
as a major challenge by 45 percent of low-performing schools. 

• The challenge least frequently rated as major was concern or opposition from parents or 
community groups about reform activities, which was rated as a major challenge by 9 percent 
of low-performing schools. 
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Table 5-3. Percentage of low-performing schools that reported major challenges when working on 
school organization and improvement: 2011-12 


School challenge 

Percent of schools 

Reported 
challenge as a 
major challenge 1 

Rating the 
challenge 2 

Restrictions in rules and regulations on replacing less effective 
teachers 

49 

76 

Insufficient funding to make substantial changes to school day or 
year schedules 

45 

73 

Insufficient funding to support special programs for students and 
families 

37 

83 

Restrictions in rules and regulations on making substantial 
changes to school day or year schedules 

36 

75 

Insufficient funding to support school-based experts (outside 
consultants, instructional specialists or coaches, mentors) 

36 

83 

Insufficient funding to purchase technology for classroom use 

32 

82 

Insufficient help from local social services and other community- 
based organizations in providing services to students and their 
families 

22 

82 

Lack of clear district guidance or support focused on staffing or 
budgeting decisions made at the school level 

20 

77 

Lack of school staff capacity or expertise to effectively use 
technology to improve instruction 

19 

82 

Lack of clear district guidance or support focused on 
implementing a whole-school intervention or turnaround model 

17 

69 

Concerns or opposition from parents or community groups about 
reform activities 

9 

75 


Table Reads: In 2011-12, 49 percent of low-performing schools rating this challenge perceived restrictions in rules and 
regulations on replacing less effective teachers as a major challenge to school organization and improvement. Seventy-six 
percent of low-performing schools rated this challenge. 

1 The challenge could be rated as not a challenge, as a minor challenge, or as a major challenge. 

2 Schools rating the challenge excludes schools that identified the challenge as not applicable, schools that did not respond to 
the question, and schools that were not low performing. 

Notes: The percentages in the table are cross-sectional estimates for the population of low-performing schools that were 
operating in the 2010-11 school year. Low-performing schools include schools that were (1) in improvement, corrective action, 
or restructuring; (2) identified as among the lowest-achieving schools; or (3) have had a graduation rate below 60 percent over 
a number of years. See appendix A for more information about the generalizability of the estimates. Detailed tables in appendix 
F provide confidence intervals for each percentage. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. Approved state applications for School Improvement grants. 
Retrieved December 2010 from http://www2.ed.gov/programs/sif/ . 
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Comparisons Across Levels 


In this section, we point out similarities and differences among the SEA, district, and school 
levels in the progress of reform implementation for improving low-performing schools and in the 
challenges experienced by those implementing these reforms. 

Reforms to Support Improvement in Low-Performing Schools: 2009-10 to 2011-12 

We examined the pattern of implementation across state, district, and school levels of two 
types of reforms for which we had related indicators at all three levels: implementation of the SIG 
school reform models and the use of compensation incentives to attract educators to low- performing 
schools. 


• While the number of SEAs providing guidance on choosing and implementing school 
intervention models increased during the period to include all 51 SEAs in 2011-12, reports 
from low-performing schools and their districts suggest that relatively little use was made of 
key elements of the SIG closure, restart, and turnaround models (figures 5-1, 5-3, and 5-4a). 

o Among districts with low-performing schools, the percentage reporting targeting schools for 
closure was 2 percent for 2011-12; the percentage contracting with external organizations 
to operate low-performing schools (expected under the SIG restart model) was 3 percent; 
and the percentage replacing the principal and teachers (expected under the SIG 
turnaround model) was 5 percent. 

o At the school level, the percentage of low-performing schools reporting that a substantial 
number of teachers were replaced was 11 percent and had not increased significantly 
between 2009-10 and 2011-12, as would be expected if more use was being made of the 
turnaround model. 

• Across levels, the implementation of compensation incentives to improve staffing at low- 
performing schools remained low during the study period (figures 5-1, 5-3, and 5-4a). 

o Between 2009-10 and 2011-12, SEAs reporting supporting compensation incentives to 
improve staffing at low-performing schools increased from 8 to 14 SEAs in 2011-12. 

o Sixteen percent of districts with low-performing schools reported in 2011-12 that they 
provided compensation incentives to improve staffing in low-performing schools. 

o Five percent of the low-performing schools reported the use of compensation incentives for 
teachers who move to teach at the school, and there had been no increase between 2009- 
10 and 2011-12. 
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Challenges Associated With Reforms to Support Improvement in Low-Performing 
Schools: 2011-12 

Because responsibilities and opportunities for improving low-performing schools differ for SEAs, 
districts with low-performing schools, and the low-performing schools themselves, each level of the 
system is likely to experience different challenges in implementing reforms to improve low-performing 
schools. For this reason, the study did not present the same list of possible challenges to SEAs, districts, 
and schools. However, challenges related to school time extension and guidance and support in 
implementing school reform models were presented to respondents across all three levels. 

• Insufficient funding to make substantial changes to the school day or year was the second 
most frequently rated major challenge for districts and schools in 2011-12. At the state level, 
restrictions in rules and regulations regarding extension of the school day or year were not 
rated among the top two major challenges by SEAs in 2011-12 (tables 5-1, 5-2, and 5-3). 

o Twelve of the 48 SEAs rating the challenge) reported that restrictions in rules and 
regulations regarding extension of the school day or year was a major challenge. 

o Among districts with one or more low-performing schools, 57 percent rated insufficient 
funding to make substantial changes to school day or year schedules as a major challenge. 

o Among low-performing schools, 45 percent rated insufficient funding to make substantial 
changes to school day or year schedules as a major challenge. 

• Relatively few SEAs, districts, or schools reported lack of guidance or support from the 
governmental level on implementing school reform models as a major challenge in 2011-12. 

o Three of the 49 SEAs rating the challenge reported that lack of clear federal guidance or 
support focused on implementing whole-school reform or turnaround models was a major 
challenge. 

o Fifteen percent of districts reported that lack of clear SEA guidance or support focused on 
adoption of whole-school reform models was a major challenge. 

o Seventeen percent of low-performing schools reported that lack of clear district guidance or 
support focused on implementing a whole-school intervention or turnaround model was a 
major challenge. 
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Chapter 6: Breadth of Reform Across Assurance Areas 


As described in the previous chapters, implementation across the reform areas varied 
considerably. More SEAs implemented reforms in the area of standards and assessment than in 
educator evaluation and compensation. District and school findings were similar to the state level for 
standards and assessments, but were less aligned in other reform areas. In the area of educator 
workforce development, across the three years surveyed, fewer districts and schools implemented 
reforms differentiating teacher compensation based on student achievement. 

Furthermore, in examining findings by selected state, district, and school characteristics, we 
found that, as expected, states that received RTT awards were more likely than other states to 
implement reforms across the assurance areas in 2011-12. 63 (Although state implementation of reforms 
in the areas of standards and assessments, and data systems was generally high across all states.) In 
keeping with expectations, large districts were significantly more likely than small and medium size 
districts to implement reforms across the assurance areas. 64 However, in contrast to expectations, the 
study found generally no significant differences between high-poverty and other districts in the 
implementation of reforms related to the assurance areas. 65 Low-performing schools were significantly 
less likely than other schools to report implementing two of the three reforms examined related to 
standards and assessments, but were significantly more likely than other schools to report using 
longitudinal data to track student achievement gains and use these gains to evaluate principals and to 
differentiate teacher compensation. 

The four assurance areas of the Recovery Act were intended to constitute an integrated, 
comprehensive vision of educational improvement. 66 Many of the reforms promoted by the act were 
intended to be mutually reinforcing. Ideally, all of these reforms would be implemented at all three 
levels of the system. Over time, more reforms in each assurance area should be implemented at each 
level, and larger numbers or percentages of states, districts, and schools should be implementing 
multiple reforms across all the assurance areas. In this chapter, we examine the extent to which states, 
districts, and schools implemented multiple reforms across assurance areas over the period of the study. 
We report on the degree to which progress in implementing the reforms varied across assurance areas 


Because states received RTT awards based, in part, on their actual and planned implementation of reforms similar to those 
included in the Recovery Act, we hypothesized that these states to be farther ahead on these reforms, compared with other 
states, by 2011-12. Large districts were hypothesized to have greater district-wide administrative capacity to support 
educational operations and educational change to help support the Recovery Act reforms than small or medium-sized 
districts. We hypothesized that high-poverty districts were more likely to be implementing reforms promoted by the Recovery 
Act given additional federal funds directly linked to incidence of poverty and experience complying with related federal 
requirements. 

64 As a reminder, the district-level analysis of reforms to support improvement in low-performing schools was limited to 
districts with low-performing schools. We did not analyze the data for districts with low-performing schools by district size 
and district poverty status. 

65 The few exceptions were that high-poverty districts were significantly more likely than other districts to differentiate teacher 
compensation based on student achievement gains, but were significantly less likely than other districts to be aware of state 
adoption of the CCSS and to provide professional development on new or revised standards for educators who teach or 
mentor mathematics or reading/ELA. 

66 While the intent of the act was that states, districts, and schools would implement reforms in all of the assurance areas, the 
report does not address whether state, district, or school implementation of more reforms constitutes an integrated and 
comprehensive reform approach. 
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and among the findings for SEAs, districts, and schools. We also report the total number of reform 
indicators met across assurance areas for each level and look across the assurance areas and determine 
whether there was variation in the reforms implemented by district and school characteristics. Finally, 
we examined challenges across the assurance areas and report on whether percentage of SEAs rating 
challenges as major differed for states that received an RTT grant compared with those that did not. 67 


Progress of Reforms Across Time and Level 


• The prevalence of Recovery Act reforms increased in all assurance areas at the state level, but 

implementation progress was slower and more uneven across districts and schools. 

o At the state level, the number of SEAs that reported implementing reforms increased 
between 2009-10 and 2011-12 in all assurance areas for all but two of the indicators the 
study tracked (table 6-1). The greatest increase among the indicators measures was SEA 
support for use of student achievement gains for principal evaluation (from 6 in 2009-10 to 
22 in 2011-12). 

o At the district level, the only assurance area with a significant increase in the percentage of 
districts that reported implementing reforms is standards and assessment, where there was 
an increase in the percentage of districts meeting three of the four indicators (table 6-2). 68 
In the educator workforce development area, there was a significant decrease in the 
percentage of districts that reported implementing reforms for three of the five indicators. 

o At the school level, for all indicators, there were significant increases over time in the 
percentage of schools that reported implementing reforms related to standards and 
assessment and data systems, but in educator workforce development, there was a 
significant increase in just one of the indicators (table 6-3). 

o Among low-performing schools, there were significant increases in the percentage of 
schools that reported implementing 7 of the 11 reforms aimed at supporting school 
improvement. 

• The most progress on the reform indicators we tracked was made in the standards and 

assessments assurance area. 

o There was an increase in the number of states meeting each indicator in this assurance area, 
a significant increase in the percentage of districts meeting three of the four indicators, and 
a significant increase in the number of schools meeting each indicator. 


67 At the district level, we examined reports of major challenges by district size and poverty status. We examined school reports 
of major challenges by performance status. These analyses did not identify clear patterns and are not part of this report. 

68 This analysis was not performed for the indicators of support for low-performing schools at the district level because the 
2011 survey did not collect data about which districts had low-performing schools in 2009-10. 
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Table 6-1. Change in the number of states that implemented reforms, by assurance area and 
indicator 


Change between 2009-10 and 2011-12 1 

Indicator by assurance area Decrease 

No change 

Increase 

Standards and assessments 

State had adopted the Common Core State Standards in 
mathematics and reading/English language arts 


X 

State was a member of a federally funded consortium developing 
assessments aligned to the Common Core State Standards 


X 

State education agency (SEA) provided, guided, or funded 
professional development on the Common Core State Standards 2 


X 

SEA provided instructional materials or curriculum assistance for the 
Common Core State Standards 2 


X 

Data systems 

SEA facilitated educators' access to assessment data 


X 

SEA provided professional development or technical assistance to 
support educators' use of assessment data 2 


X 

State data system had ability to link teachers to student data 

X 


State operated a longitudinal data system that included 12 core 
components 

X 


Educator workforce development 

SEA simplified/shortened educator licensure process or authorized 
non-university preparation programs 2 


X 

SEA issued standards or guidelines for teacher preparation programs 2 


X 

SEA issued standards or guidelines for principal preparation 
programs 2 


X 

SEA supported differentiating teacher compensation based on 
student achievement gains 3 


X 

SEA supported use of student achievement gains for principal 
evaluation 3 


X 

SEA supported use of multi-level ratings, multiple observations, and 
student achievement gains for teacher evaluation 3 


X 


continued 
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Table 6-1. Change in the number of states that implemented reforms, by assurance area and 
indicator (cont'd) 



Change between 2009-10 and 2011-12 1 

Indicator by assurance area 

Decrease No change Increase 

Support for low-performing schools 

SEA provided guidance on choosing and implementing school 
intervention models defined by ED 2 

X 

State allowed for expansion of the number of charter schools 4 

X 

SEA supported using compensation incentives to improve staffing at 
low-performing schools 3 

X 

SEA monitored deployment of effective educators in low-performing 
schools 

X 


Table reads: The number of SEAs that adopted the Common Core State Standards in mathematics and reading/English language 
arts increased from 2010-11 to 2011-12. 

1 For the standards and assessments indicators, change is from 2010-11 to 2011-12 because before 2010-11, the Common Core 
State Standards were not yet available. For two data systems indicators— state data system had ability to link teachers to 
student data and state operated a longitudinal data system that included 12 core components— change is from 2009-10 to 
2010-11 because the data were not available for 2011-12. For all other assurance areas, change is from 2009-10 to 2011-12. 

2 In measuring state reform activity for this indicator, if an SEA reported that it took action in a previous year, the state is 
counted as meeting the indicator in subsequent years. 

3 To meet this indicator, SEAs not only had to play a role in supporting implementation, but also had to be supporting the 
specific types of systems that the Recovery Act explicitly identified. 

4 In measuring state reform activity for this indicator, if in a previous year a state passed legislation to either increase the 
permissible number of charter schools or remove prohibitions on charter schools, the state is counted as meeting the 
indicator in subsequent years. 

Note: Respondents include 50 states and DC. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 State Education Agency Surveys and, for information on the 
expansion of charter schools, National Alliance for Public Charter Schools report: Measuring Up to the Model: A Ranking of 
State Charter School Laws (2010 and 2011). 
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Table 6-2. Change in the percentage of districts that implemented reforms, by assurance area and 
indicator 1 



Change between 2009-10 and 2011-12 2 

Indicator by assurance area 

Significant 

decrease 

No 

significant 

change 

Significant 

increase 

Standards and assessments 

District aware of state adoption of Common Core State Standards 



X 

District provided professional development on new or revised state 
content standards on instructional strategies for teachers to help 
English learners or students with disabilities master the content 
standards 


X 


District provided professional development on new or revised state 
content standards for educators who teach or mentor mathematics 
or reading/English language Arts 



X 

District distributed instructional materials or provided selection 
guidance on curricula aligned with the new or revised state content 
standards 



X 

Data systems 

District provided educators with access to assessment data 


X 


District provided educators with professional development on the 
use of assessment data for instructional planning 


X 


District used longitudinal data to track student achievement gains 
for individual teachers 


X 


Educator workforce development 

District provided school leaders with professional development or 
flexibility to hire effective teachers 

X 



District used student achievement gains for teacher tenure, 
dismissal, or assignment decisions 

X 



District operated a principal evaluation system that included 
student achievement gains 


X 


District operated a teacher evaluation system that included multi- 
level rubrics, multiple observations, and student achievement gains 


X 


District differentiated teacher compensation based on student 
achievement gains 

X 




Table Reads: The percentage of districts that were award of state adoption of the Common Core State Standards increased 
significantly from 2010-11 to 2011-12. 

1 This analysis does not include indicators of support for low-performing schools at the district level because the 2011 survey 
did not collect data about which districts had low-performing schools in 2009-10. 

2 For the standards and assessments indicators, change is from 2010-11 to 2011-12 because before 2010-11, the Common Core 
State Standards were not yet available. For all other assurance areas, change is from 2009-10 to 2011-12. 

Notes: The change status is based on cross-sectional estimates for the population of districts that were operating in the 2010- 
11 school year. See appendix A for more information about the generalizability of the estimates. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 District Surveys. 
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Table 6-3. Status of change in the percentage of schools that implemented reforms, by assurance 
area and indicator 



Change between 2009-10 and 2011-12 1 

Indicator by assurance area 

Significant 

decrease 

No 

significant 

change 

Significant 

increase 

Standards and assessments 

Teachers received professional development on new or revised state 
content standards 



X 

Teachers received professional development targeted to help English 
learners or students with disabilities master new or revised state content 
standards 



X 

School used curriculum or curriculum materials aligned with new or 
revised state content standards 



X 

Data systems 

School used student assessment data to identify students for additional 
support 



X 

Teachers had online access to student assessment results 



X 

School used student assessment data to tailor instruction 



X 

School used longitudinal data to track student achievement gains for 
individual teachers 



X 

Educator workforce development 

Principal evaluation practices included student achievement gains 


X 


Teacher tenure, dismissal, or reassignment decisions used student 
achievement gains 



X 

Teacher evaluation practices included multi-level rubrics, multiple 
observations, and student achievement gains 


X 


Teacher compensation differentiated based on student achievement 
gains 


X 



continued 
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Table 6-3. Status of change in the percentage of schools implementing reforms, by assurance area 
and indicator (cont'd) 



Change between 2009-10 and 2011-12 

Indicator by assurance area 

Significant 

decrease 

No 

significant 

change 

Significant 

increase 

Improvement strategies in place 2 

Nonfinancial incentives to encourage teachers to move to or remain in 
the school 


X 


Substantial portion of teachers replaced as part of school restructuring 


X 


Compensation incentives for teachers who move to teach at the school 


X 


Effective teachers reassigned to school as part of school restructuring 


X 


School modified daily schedule to increase instructional time for 
reading/English language arts or mathematics 



X 

School used outside school improvement experts as part of school 
restructuring or to improve instruction 



X 

School day, week, or year extended as part of school restructuring 



X 

Staffing or budgeting autonomy increased as part of school restructuring 



X 

School implemented programs to encourage family and community 
involvement 



X 

School implemented programs to address students' social and emotional 
needs 



X 

School implemented programs to orient parents to school improvement 
models 



X 


Table Reads: There was no significant change from 2010-11 to 2011-12 in the percentage of schools that reported that teachers 
received professional development on new or revised state content standards. 

1 For the standards and assessments indicators, change is from 2010-11 to 2011-12 for districts in states that adopted Common 
Core State Standards. Before 2010-11, the Common Core State Standards were not available. For all other assurance areas, 
change is from 2009-10 to 2011-12. 

2 For the support for improvement indicators, change is based only on low-performing schools. 

Notes: The change status is based on cross-sectional estimates for the population of schools that were operating in the 2010-11 
school year. See appendix A for more information about the generalizability of the estimates. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. 
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Breadth of Reform at the State Level 


The more reforms a state implemented, the more that state is likely to be moving toward a 
comprehensive program of reform consistent with the act's vision. We examined the breadth of reform 
by counting the total number of indicators met by each state. 

• States vary widely in the number of reforms they have implemented. Two years after 
enactment of the Recovery Act, implementation of the reform indicators ranged from two 
states meeting all 18 indicators to two states meeting 5 of the 18 (figure 6-1). 

• The median number of indicators met by 2011-12 was 11 of 18. 
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Figure 6-1. Total number of reform indicators met by state, 2009-10 to 2011-12 
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Figure Reads: In 2009-10, Florida met 12 reform indicators. By 2011-12, Florida met six more reform indicators. Thus, Florida 
met all possible reform indicators by 2011-12. 

Note: The maximum number of indicators was 14 in 2009-10 and 18 in 2011-12. For two data systems indicators, data were not 
available for 2011-12. The change by 2011-12 reflects the status of these indicators in 2010-11. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 State Education Agency Surveys. National Alliance for Public 
Charter Schools report: Measuring Up to the Model: A Ranking of State Charter School Laws (2010 and 2011). 
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Breadth of Reform at the District and School Levels, 2011-12 


As at the state level, the more reforms implemented by districts and schools, the more fully the 
act's reform agenda is likely to be realized. We examined the breadth of reform at the district and 
school levels by charting the percentages of districts and schools that met different numbers of the 
indicators of reform in 2011-12, from all to none. 

Because districts without low-performing schools and schools that were not low performing 
would not be expected to implement many of the reforms related to the low-preforming schools 
assurance area, we looked first at the subset of indicators that apply to all schools and districts (those 
from the standards and assessment, data systems, and educator workforce assurance areas) and then 
separately at those related to supporting low-performing schools. We also examined the results from 
the district and school subgroup analyses across assurance areas to determine if there are patterns in 
the types of districts and schools implementing reforms in 2011-12. 

Findings 

• Only a small minority of districts and schools (5 percent or fewer) implemented all or almost 
all of the reforms we examined related to standards and assessment, data systems, and 
educator workforce development. 69 

o One percent of districts met all indicators from these three assurance areas, and 3 percent 
met at least 11 of the 12 (figure 6.2). 

o The median number of indicators met by districts across these assurance areas was 6 of the 
12 we examined. 

o One percent of schools met all of the indicators, and 4 percent of schools met 10 of the 11 
(figure 6.3). 

o The median number of indicators met by schools was 6 of 11. 

• Only a small percentage of districts with low-performing schools (less than 2 percent) met all 
or almost all of the indicators representing reforms aimed at supporting school improvement 
(figure 6-4). 

o Less than 1 percent of these districts met all indicators in this assurance area, and 1 percent 
met at least eight of the nine indicators. 

o Seventeen percent of these districts did not meet any of the indicators relating to 
supporting low-performing schools. 

o The median number of indicators pertaining to low-performing schools that these districts 
met was two of the possible nine. 


69 Appendix tables G.l and G2 provide the percentage of districts and schools that implemented all or almost all of the reforms 
in 2009-10. 
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• Among the low-performing schools, less than 1 percent met all the indicators pertaining to 
school improvement in 2011-12, and 1 percent met at least 10 of the 11 indicators 
(figure 6-5). 70 

o Eight percent of these schools met none of the indicators, 
o The median number of indicators implemented was 5 of the 11 we examined. 

• Large districts were significantly more likely than medium and small districts to report that they 
were implementing reforms in 2011-12 in the standards and assessments, data systems, and 
educator workforce development assurance areas. For all but one reform indicator, a greater 
percentage of large districts met the indicator than medium and small districts (figures 2-5, 3-5, 
and 4-5). 

• There were no consistent patterns in reform implementation across the assurance areas by 
district poverty and low-performing school status. 


70 Appendix table G.5 provides the percentage of low performing schools that implemented all of the indicators pertaining to 
school improvement in 2009-10. 
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Figure 6-2. Percentage of districts that implemented reforms in standards and assessments, data 

systems, and educator workforce development in 2011-12, by number of indicators met 



Percent of All Districts 


Figure Reads: In 2011-12, 1 percent of districts met all 12 reform indicators in the areas of standards and assessments, data 
systems, and educator workforce development. 

Notes: The percentages in the figure are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. Detailed tables in 
appendix G provide confidence intervals for each percentage. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Figure 6-3. Percentage of schools that implemented reforms in standards and assessments, data 

systems, and educator workforce development in 2011-12, by number of indicators met 



Percent of All Schools 

Figure Reads: In 2011-12, 1 percent of schools met all 11 reform indicators in the areas of standards and assessments, data 
systems, and educator workforce development. 

Notes: The percentages in the figure are cross-sectional estimates for the population of schools that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. Detailed tables in 
appendix G provide confidence intervals for each percentage. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Figure 6-4. Percentage of districts with low-performing schools that implemented school 
improvement reforms in 2011-12, by number of indicators met 



Percent of Districts with Low-Performing Schools 


Figure Reads: In 2011-12, less than 1 percent of districts with low-performing schools met all nine reform indicators in the 
area support for low-performing schools. 

Notes: The percentages in the figure are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year and have low-performing schools. See appendix A for more information about the generalizability of the 
estimates. Detailed tables in appendix G provide confidence intervals for each percentage. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Figure 6-5. Percentage of low-performing schools that supported school improvement in 2011-12, by 
number of indicators met 



Figure Reads: In 2011-12, less than 1 percent of low-performing schools met all 11 reform indicators in the area of school 
improvement. 

Notes: The percentages in the figure are cross-sectional estimates for the population of low-performing schools that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. 
Detailed tables in appendix G provide confidence intervals for each percentage. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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State Challenges by Race to the Top Status, 2011-12 


The study also investigated whether the percentage of SEAs rating challenges as major differed 
for states that received an RTT grant compared with those that did not. 71 We hypothesized that a 
smaller percentage of SEAs in RTT states would rate challenges as major in 2011-12 because they were 
more prepared to respond to challenges due to their prior experience implementing the reforms 72 and 
the resources provided by the grant. 


Findings 

• In the area of standards and assessments, with the exception of challenges not perceived as 
major by any SEA, smaller percentages of SEAs in RTT states rated challenges as major 
challenges compared with SEAs in other states that adopted CCSS (table 6-4). Differences 
ranged from 3 to 32 percentage points. 

• When implementing data systems reforms, similar or smaller percentages of SEAs in RTT 
states rated challenges using data to support reform as major challenges compared with SEAs 
in other states (table 6-5). Differences ranged from 1 to 31 percentage points. 

• Most educator workforce reform challenges also were rated as major by a lower percentage 
of RTT SEAs (by 1 to 29 percentages points), with one notable exception (table 6-6). 

o SEAs in RTT states were considerably more likely than SEAs in other states to report lack of 
SEA staff or expertise to develop reliable and fair methods for a statewide system of 
educator performance evaluation based partly on student achievement as a major challenge 
(30 percentage points higher). 

• In contrast to the findings above, in the support for improving low-performing schools area, 
similar or larger percentages of SEAs in RTT states rated challenges as major compared with 
SEAs in other states (differences ranged from 1 to 21 percentage points), with one exception 
(table 6-7). 

o SEAs in RTT states were less likely than SEAs in other states to report concerns or opposition 
from educators about closing or restructuring schools (13 percentage points lower). 


71 At the district level, we examined reports of major challenges by district size and poverty status. We examined school reports 
of major challenges by performance status. These analyses did not identify clear patterns and are not part of this report. 

72 See table 6-1 of Webber, A., Troppe, P., Milanowski, A., Gutmann, B., Reisner, E., and Goertz, M. (2014). 
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Table 6-4. Percentage of state education agencies (SEAs) in states that adopted the Common Core 
State Standards (CCSS) that reported major challenges when implementing new or 
revised state content standards and aligned assessments, by RTT status: 2011-12 


SEA challenge 

SEAs that reported challenge as a 
major challenge 1 

Applicable SEAs 2 

RTT states 

Other states 

RTT states 

Other states 

Lack of SEA staff or expertise to provide 
districts with professional development or 
technical assistance on developing interim or 
formative assessments to measure student 
mastery of the new or revised state content 
standards 

40% 
(N =4 of 10) 

55% 

(N = 12 of 22) 

83% 

(N = 10 of 12) 

65% 

(N = 22 of 34) 

Lack of instructional materials aligned with the 
new or revised state content standards 

45% 
(N = 5 of 11) 

48% 

(N = 14 of 29) 

92% 

(N = 11 of 12) 

85% 

(N = 29 of 34) 

Lack of SEA staff or expertise to provide 
districts with professional development or 
technical assistance on developing 
instructional materials aligned with the new or 
revised state content standards 

25% 
(N = 3 of 12) 

50% 

(N = 13 of 26) 

100% 
(N = 12 of 12) 

76% 

(N = 26 of 34) 

Lack of assessments to measure student 
mastery of the new or revised state content 
standards 

18% 

(N = 2 of 11) 

50% 

(N = 12 of 24) 

92% 

(N = 11 of 12) 

71% 

(N =24 of 34) 

Lack of SEA staff or expertise to provide 
districts with professional development or 
technical assistance on implementing new or 
revised state content standards 

17% 
(N = 2 of 12) 

45% 

(N = 14 of 31) 

100% 
(N = 12 of 12) 

91% 

(N =31 of 34) 

Lack of SEA staff or expertise to provide 
districts with professional development or 
technical assistance on implementing new 
state assessments 

20% 
(N = 2 of 10) 

41% 
(N =7 of 17) 

83% 

(N = 10 of 12) 

50% 

(N = 17 of 34) 

Opposition from educators or other groups to 
the new or revised state content standards 

0% 

(N = 0 of 12) 

0% 

(N = Oof 31) 

100% 
(N = 12 of 12) 

91% 

(N =31 of 34) 

Opposition from educators or other groups to 
the new or revised state assessments 

0% 

(N = 0of 11) 

0% 

(N = Oof 23) 

92% 

(N = 11 of 12) 

68% 

(N = 23 of 34) 


Table Reads: In 2011-12, 40 percent of RTT SEAs in CCSS states rating this challenge perceived lack of SEA staff or expertise to 
provide districts with professional development or technical assistance on developing interim or formative assessments to 
measure student mastery of the new or revised state content standards as a major challenge to implementing new or revised 
state standards and aligned assessments. Ten RTT SEAs in CCSS states rated this challenge. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Applicable SEAs are those that rated the challenge. This excludes SEAs that identified the challenge as not applicable, SEAs 
that did not respond to the question, and SEAs that did not adopt the CCSS in both reading/English language arts and 
mathematics. 

Note: Respondents include 50 states and DC. RTT states are the 12 states awarded RTT grants in the first two rounds of 
competition (DE, DC, FL, GA, HI, MA, MD, NC, NY, OH, Rl, TN). Other states include only the 34 non-RTT states that adopted 
the CCSS in mathematics and reading/English language arts. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey. 
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Table 6-5. Percentage of state education agencies (SEAs) that reported major challenges using data 
to support reform, by RTT status: 2011-12 



SEAs that reported challenge as a 




major challenge 1 

Applicable SEAs 

SEA challenge 

RTT states 

Other states 

RTT states 

Other states 

Restrictions in rules and regulations on linking of 

10% 

41% 

83% 

82% 

student data to individual teachers 

(N = 1 of 10) 

(N = 13 of 32) 

(N = 10 of 12) 

(N =32 of 39) 

Current data systems make: 





Linking student test data to individual teachers 

33% 

31% 

100% 

92% 

difficult 

(N =4 of 12) 

(N = 11 of 36) 

(N = 12 of 12) 

(N =36 of 39) 

Tracking the success of school improvement 

25% 

23% 

100% 

100% 

efforts at the student level difficult 

(N =3 of 12) 

(N =9 of 39) 

(N = 12 of 12) 

(N =39 of 39) 

Current data systems limit LEA and school 

9% 

22% 

92% 

69% 

access to new assessment data 

(1 of 11) 

(6 of 27) 

(11 of 12) 

(27 of 39) 

Lack of SEA staff or expertise to provide districts 
with professional development or technical 
assistance on accessing and using assessment data 

9% 

(N = 1 of 11) 

26% 
(N =8 of 31) 

92% 

(N = 11 of 12) 

79% 

(N =31 of 39) 


Table Reads: In 2011-12, 10 percent of RTT SEAs rating this challenge perceived restrictions in rules and regulations on linking of 
student data to individual teachers as a major challenge to using data to support reform. Ten RTT SEAs rated this challenge. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Applicable SEAs are those that rated the challenge. This excludes SEAs that identified the challenge as not applicable and SEAs that 
did not respond to the question. 

Note: Respondents include 50 states and DC. RTT states are the 12 states awarded RTT grants in the first two rounds of competition 
(DE, DC, FL, GA, HI, MA, MD, NC, NY, OH, Rl, TN). 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An Evaluation of 
the Recovery Act's Role: Spring 2012 State Education Agency Survey. 
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Table 6-6. Percentage of state education agencies (SEAs) that reported major challenges when 

working with districts and others to develop and manage a skilled educator workforce, by 
RTT status: 2011-12 


SEA challenge 

SEAs that reported challenge 
as a major challenge 1 

Applicable SEAs 2 

RTT states 

Other states 

RTT states 

Other states 

Difficulty in measuring student growth for teachers 
of non-tested subjects 

75% 
(N = 9 of 12) 

76% 

(N = 26 of 34) 

100% 
(N = 12 of 12) 

87% 

(N =34 of 39) 

Concerns or opposition from educators about 
performance-based compensation 

50% 
(N = 5 of 10) 

79% 

(N = 19 of 24) 

83% 

(N = 10 of 12) 

62% 

(N = 24 of 39) 

Concerns or opposition from educators about 
evaluating educators based, at least in part, on 
student achievement 

50% 
(N = 6 of 12) 

51% 

(N = 18 of 35) 

100% 
(N = 12 of 12) 

90% 

(N =35 of 39) 

Lack of SEA staff or expertise to provide LEAs with 
professional development or technical assistance 
on differentiated teacher compensation systems 

30% 
(N = 3 of 10) 

55% 

(N = 11 of 20) 

83% 

(N = 10 of 12) 

51% 

(N = 20 of 39) 

Lack of SEA staff or expertise to develop reliable 
and fair methods for a statewide system of 
educator performance evaluation based partly on 
student achievement 

64% 
(N = 7 of 11) 

34% 

(N = 12 of 35) 

92% 

(N = 11 of 12) 

90% 

(N = 35 of 39) 

Restrictions in rules and regulations on how 
educators can be compensated 

33% 
(N =3 of 9) 

39% 
(N =9 of 23) 

75% 
(N = 9 of 12) 

59% 

(N = 23 of 39) 

Lack of SEA staff or expertise to provide LEAs with 
professional development or technical assistance 
on educator recruitment, hiring, and induction 

27% 
(N =3 of 11) 

37% 

(N = 11 of 30) 

92% 

(N = 11 of 12) 

77% 

(N = 30 of 39) 

Resistance from colleges and universities to 
modifying educator preparation programs to 
changing state reform priorities 

27% 
(N = 3 of 11) 

25% 
(N =9 of 36) 

92% 

(N = 11 of 12) 

92% 

(N =36 of 39) 

Lack of clear federal guidance or support on 
educator compensation or evaluation systems 

18% 

(N =2 of 11) 

19% 
(N =6 of 31) 

92% 

(N = 11 of 12) 

79% 

N = (31 of 39) 

Restrictions in rules and regulations on how 
educators can be evaluated 

18% 

(N =2 of 11) 

13% 
(N =4 of 32) 

92% 

(N = 11 of 12) 

82% 

(N = 32 of 39) 


Table Reads: In 2011-12, 75 percent of RTT SEAs rating this challenge perceived difficulty in measuring student growth for 
teachers of non-tested subjects as a major challenge when working with districts and others to develop and manage a skilled 
educator workforce. Twelve RTT SEAs rated this challenge. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Applicable SEAs are those that rated the challenge. This excludes SEAs that identified the challenge as not applicable and SEAs 
that did not respond to the question. 

Note: Respondents include 50 states and DC. RTT states are the 12 states awarded RTT grants in the first two rounds of 
competition (DE, DC, FL, GA, HI, MA, MD, NC, NY, OH, Rl, TN). 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey. 
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Table 6-7. Percentage of state education agencies (SEAs) that reported major challenges in efforts to 
support improvement in low-performing schools, by RTT status: 2011-12 


SEA challenge 

SEAs that reported challenge as 
a major challenge 1 

Applicable SEAs 2 

RTT states 

Other states 

RTT states 

Other states 

Restrictions in rules and regulations regarding 
extent of autonomy that local education 
agencies and schools can be granted in terms of 
staffing or budgets 

42% 
(N =5 of 12) 

27% 

(N = 10 of 37) 

100% 
(N = 12 of 12) 

95% 

(N =37 of 39) 

Restrictions in rules and regulations regarding 
teacher hiring practices 

45% 
(N = 5 of 11) 

24% 
(N =8 of 34) 

92% 

(N = 11 of 12) 

87% 

(N = 34 of 39) 

Concerns or opposition from educators about 
closing or restructuring schools 

18% 
(N = 2 of 11) 

31% 

(N = 11 of 36) 

92% 

(N = 11 of 12) 

92% 

(N = 36 of 39) 

Restrictions in rules and regulations regarding 
extension of school day or year 

33% 
(N = 4 of 12) 

22% 
(N =8 of 36) 

100% 
(N = 12 of 12) 

92% 

(N = 36 of 39) 

Restrictions in rules and regulations regarding 
number of schools that can be closed, opened as 
charters, or restructured in other ways 

8% 

(N = 1 of 12) 

12% 
(N =3 of 26) 

100% 
(N = 12 of 12) 

67% 

(N = 26 of 39) 

Lack of SEA staff or expertise to provide 
guidance and technical assistance on whole- 
school reform or turnaround models to LEAs 

8% 

(N = 1 of 12) 

10% 
(N =4 of 39) 

100% 
(N = 12 of 12) 

100% 
(N =39 of 39) 

Lack of SEA staff or expertise to provide 
professional development focused on improving 
low-performing schools 

9% 

(N = lof 11) 

8% 

(N =3 of 39) 

92% 

(N = 11 of 12) 

100% 
(N = 39 of 39) 

Lack of clear federal guidance or support focused 
on implementing whole-school reform or 
turnaround models 

9% 

(N = lof 11) 

5% 

(N = 2 of 38) 

92% 

(N = 11 of 12) 

97% 

(N = 38 of 39) 

Lack of SEA staff or expertise to identify and 
disseminate best practices concerning improving 
low-performing schools 

9% 

(N = lof 11) 

5% 

(N =2 of 39) 

92% 

(N = 11 of 12) 

100% 
(N = 39 of 39) 

Lack of SEA staff or expertise to screen and 
disseminate information on EMOs, CMOs and 
school turnaround experts 2 

9% 

(N = lof 11) 

0% 

(N = Oof 21) 

92% 

(N = 11 of 12) 

54% 

(N =21 of 39) 


Table Reads: In 2011-12, 42 percent of RTT SEAs rating this challenge perceived restrictions in rules and regulations regarding 
extent of autonomy that local education agencies and schools can be granted in terms of staffing or budgets in efforts to 
support improvement in low-performing schools. Twelve RTT SEAs rated this challenge. 

1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Applicable SEAs are those that rated the challenge. This excludes SEAs that identified the challenge as not applicable and SEAs 
that did not respond to the question. 

Note: Respondents include 50 states and DC. RTT states are the 12 states awarded RTT grants in the first two rounds of 
competition (DE, DC, FL, GA, HI, MA, MD, NC, NY, OH, Rl, TN). 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agency Survey. 
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This methodology appendix summarizes the data sources for the study, describes the district 
and school sample designs, and presents the survey response rates. It also includes a description of 
weighting and nonresponse adjustments. In addition, it reviews the study team's approach to the 
analyses presented in this report including a discussion of indicator construction and statistical testing. 

Data Sources 

The analyses conducted for this report primarily used data collected during spring 2011 and 
spring 2012 through surveys administered to all 50 SEAs and the District of Columbia (DC) and nationally 
representative samples of school districts and schools. We also used extant data for three SEA 
indicators, and to classify districts and schools for sampling and analysis. In this section, we review the 
survey development process and the sources of extant data for the three SEA indicators, and the 
characteristics of districts and schools for the subgroup analyses 

Survey Development Process 

The spring 2011 surveys asked SEAs, districts, and schools about activities in the 2009-10 and 
2010-11 school years. The spring 2012 surveys asked them about activities in the 2011-12 school year. 

Development of the survey instruments occurred in three-stages. During the first stage, the 
study's survey design team drafted initial survey items based on documents describing the Recovery 
Act's constituent programs and research on reforms related to the four assurance areas. During the 
second stage, the design team vetted the draft survey instrument with individuals from the U.S. 
Department of Education (ED) and the study's Technical Work Group (TWG). The surveys also were 
pretested with a group of SEA and district officials and school principals. In the final stage, the design 
team incorporated feedback and refined the survey accordingly. Each stage is discussed below. 

Stage one: Drafting survey items 

During the first stage of survey development, the study team drafted individual survey items 
guided by the following goals: 

• The SEA survey should collect information on state policies and programs designed to 
support school reform activities promoted by programs funded by the Recovery Act. The 
SEA survey also should capture information on the variety of roles through which states 
could carry out the policies and programs; specific elements of their states' evaluation, 
compensation, and data systems; supports to LEAs implementing reforms; and potential 
challenges that SEAs may face when implementing reform activities. 

• The district survey should collect information on district adoption of specific education 
policies, use of strategies to support and promote reform policies at the school level, and 
potential challenges that districts may face when implementing reform strategies. 

• The school survey should collect information on whether specific practices or strategies 
associated with implementing state or district policies or programs related to the assurances 
were being used in schools. The school should also capture potential challenges school 
leaders may have encountered when implementing specific practices. 
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• The surveys should not ask SEAs, districts, or schools to provide information that was 
already available through other surveys or reporting requirements. 

With these goals in mind, the study team conducted in-depth reviews of ED documents (e.g., 
grant notices, regulations, and guidance) describing the aims and requirements for each Recovery Act 
funded program. Draft survey items included the specific strategies and activities described in SFSF 
assurance indicators and descriptors, the RTT selection criteria, and guidance for the Title l-ARRA and 
School Improvement Grant (SIG programs . 73 We also reviewed survey items included in an earlier ED- 
sponsored national study of Title I to identify any questions about education reform that, if included in 
our survey, would yield longitudinal data on a particular reform activity. 

While the goal was for the state, district, and school surveys was to cover parallel topics, 
individual survey items were tailored to focus on activities most relevant to a particular level. For 
example, in the case of standards and assessments, the SEA survey asked about state adoption of the 
Common Core State Standards and other new or revised content standards, and on specific state 
activities that supported the implementation of those content standards (e.g., professional 
development, instructional materials, and assistance to districts in curriculum mapping). The district 
survey, in turn, asked about the distribution of instructional materials to schools and whether the 
district made available or provided professional development on the new or revised state content 
standards to educators. The school survey asked what practices the school used to implement new or 
revised state content standards such as educators' receipt of professional development, use of 
curriculum frameworks, or use of curriculum aligned with the new or revised state content standards. 

The design team included nationally recognized experts with a thorough understanding of the 
latest research in the areas of educator workforce development, low-performing schools and the Title I 
program in particular, and public school finance. Members of the design team also contributed their 
experience conducting other national surveys of how states and districts were using Recovery Act 
education funds (e.g., state surveys conducted by the Center on Education Policy) and provided insight 
into the variety of roles states assume to support education reform activities. 

The team contributed this deep knowledge base to the drafting of survey items. In particular, 
the team ensured that the SEA survey captured a range of state roles, from more prescriptive roles (such 
as developing and administering a statewide evaluation system) to those that involve a more supporting 
role (such as providing technical assistance to school districts). The team also made certain that the 
district survey included the range of district policies and strategies related to implementing reforms 
promoted by the Recovery Act. For example, a few of the act's component programs bypassed the state 
level to provide funds directly to districts promising to undertake specific reforms that were related to 
the assurances. 


73 See: U.S. Department of Education. (2010). Overview Information; Race to the Top Fund; Notice Inviting Applications for New 
Awards for Fiscal Year (FY) 2010. Federal Register, 75(71). Available from http://www.gpo.gov/fdsys/pkg/FR-2009-ll- 
18/pdf/E9-27427.pdf . U.S. Department of Education. (2009, November). State Fiscal Stabilization Fund Program; Final Rule. 
Federal Register, 74(217): 58436-58525. Available from http://www2.ed.gov/legislation/FedRegister/finrule/2009- 
4/111209a.pdf. U.S. Department of Education. (2010). School Improvement Grants; American Recovery and Reinvestment 
Act of 2009 (ARRA); Title I of the Elementary and Secondary Education Act of 1965, as Amended (ESEA). Federal Register, 
75(208): 66363-66371. Available from http://www.gpo.gov/fdsvs/pkg/FR-2010-10-28/pdf/201Q-27313.pdf 


A-2 


To ensure that the survey did not duplicate other reporting requirements for the same time 
period, the study team reviewed closely the specific reporting requirements for recipients of each type 
of Recovery Act grant (e.g., SFSF, RTT). 

Stage two: Vetting the draft survey instruments 

In stage two, IES and members of the study's TWG reviewed the draft survey instruments, and 
the study team pretested the surveys with a group of SEA and district officials and school principals. An 
important insight gained during this stage was the importance of not wording survey items using 
language closely tied to language in the act and its programs (so as not to create a checklist based 
directly on the program requirements), in order to decrease the likelihood that response options would 
elicit socially desirable responses. 

We pretested each survey with no more than nine respondents, focusing on (1) wording clarity, 
(2) information availability, (3) response burden, and (4) survey administration effectiveness. The design 
team conducted debriefing discussions with pretest participants to hear their comments on these four 
topics. We pretested the SEA survey with SEA officials in three states, which included states that did and 
did not win an RTT grant in the second round of that competition. The district pretest included seven 
districts, which included a mix of rural, suburban, and urban districts. Locations included the Northwest, 
Midwest, Mid-Atlantic and Southeast. The school pretest included five elementary schools, which 
included a mix of elementary, middle, and high schools. Locations were in the Midwest and Northeast 
and included schools in large urban and suburban districts. The pretest also included schools 
participating in SIG. Several others served high-poverty populations. In response to debriefing 
discussions with survey pretest participants, we revised some survey items and instructions. The pretest 
comments also suggested a need to streamline instructions in some places and provide definitions for 
key terms. 

Stage three: Refine and finalize the survey instruments 

In the final stage of survey development, the design team carefully considered the feedback 
received from all sources and reviewed each survey item to determine if particular items or instructions 
needed revisions. The team refined the instrument as necessary. 

For the follow-up survey administration, we revised the baseline year surveys to ask about the 
2011-12 school year, to remove items no longer needed (e.g., questions related to the previous school 
year) and to clarify or eliminate items respondents appeared to have difficulty answering. We based 
these changes on preliminary reviews of the baseline survey data, questions and comments from district 
and school respondents during data collection, and SEA remarks in response to follow-up inquiries about 
baseline survey responses. The revised surveys were reviewed by IES. A TWG member, who was also an 
SEA official, reviewed the SEA follow-up survey for clarity. We refined the instruments as necessary to 
incorporate their comments. IES reviewed and approved the final changes. 

Extant data 

In this report, three SEA indicators of reform rely on extant data sources. Two of the SEA data 
systems indicators use the U.S. Department of Education, State Fiscal Stabilization Fund Initial Annual 
State Reports (2009-10) and Amended Applications (2010-11). These are the indicators of whether the 
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state data system had ability to link teachers to student data and whether the state operated a 
longitudinal data system that included the 12 core components from the America COMPETES Act. Data 
for whether the state allowed for expansion of the number of charter schools come from the National 
Alliance for Public Charter Schools report: Measuring up to the model: A ranking of state charter school 
/ows (2010 and 2011). 

The study team also used extant data sources for the selected district and school characteristics 
examined in this report. At the district level, we examined reform implementation by district size and 
district poverty status. District size was based on enrollment data from the National Center for 
Education Statistics, Common Core of Data. Local Education Agency Universe Survey: School Year 2008- 
09 (ag081a.sas7bdat). 74 Large districts have at least 50,000 students. Medium districts have fewer than 
50,000, but at least 15,000 students. Small districts have fewer than 15,000 students. District poverty 
status was based on child poverty rates from the 2008 Small Area Income and Poverty Estimates (SAIPE) 
data (USSD08.xls) from the U.S. Census Bureau. High poverty districts include all districts with poverty 
rates over 21.6 percent. Twenty-five percent of students were enrolled in schools in districts with 
poverty rates above this percentage. 75 

At the school level, for three of the four assurance areas, we also compared reform 
implementation in schools identified as low-performing with schools not so identified. School 
performance data came from approved state applications for SIG for school performance status. 76 Low- 
performing schools include schools that were (1) identified as among the lowest achieving schools; (2) 
have had a graduation rate below 60 percent over a number of years; or (3) other schools in 
improvement, corrective action, or restructuring under Title I, ESEA or identified as a tier I, II, or III 
school in the state's FY 2010 SIG application. 


Sample Design 

District Sample Design 

We selected a nationally representative sample of 1,700 districts using a student enrollment 
driven design where districts with more students enrolled were more likely to be selected for 
participation. The district sample design was similar to that used in prior U.S. Department of Education 
studies of policy implementation. 77 

The frame for the district sample was constructed using the 2008-09 Common Core of Data (CCD) 
district data with updates from the preliminary data for the 2009-10 school year. The frame included 
only regular school districts. That is, it excluded districts that had no open, functioning schools; districts 
with zero enrollment; and districts exclusively for special groups of students (e.g., children in 
correctional facilities). The final district frame included 15,632 districts, with a total enrollment of 


74 Retrieved August 24, 2010, from http://nces.ed.gov/ccd/pubagencv.asp 

75 Retrieved November 11, 2010, from http://www.census.gov/did/www/saipe/data/schools/index.html 

76 Retrieved December 2010 from http://www2.ed.gov/programs/sif/ 

77 For example, for the National Longitudinal Study of NCLB (NLS-NCLB), a total of 300 districts were sampled. For NLS-NCLB and 
this study, the samples were drawn using a probability proportionate to size approach, with the size measure being district 
enrollment. (See U.S. Department of Education, 2008). 
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48,856,050 students. The frame covered the 50 states and the District of Columbia (no US territories), 
but excluded Department of Defense and Bureau of Indian Education schools. 

We stratified the district frame by the district's high-poverty status 78 . Within the high-poverty 
stratum and its complement stratum, there were geographic strata. Districts in states that were 
awarded a Race to the Top (RTT) grant during the first two rounds of the grant competition were 
stratified by individual state so that each RTT state would be represented in the district sample. Districts 
in RTT finalist states and districts in other states were stratified by Census region (Central, Northeast, 
South, and West) to ensure balance broadly across the U.S. 79 

Districts in the high-poverty stratum were oversampled. Oversampling was needed to ensure that 
there were enough high-poverty districts to support subgroup analyses for this population of interest. 
High-poverty stratum districts are roughly one-quarter of the population of districts but with 
oversampling were roughly one-half of the sample. This allows for more precision (i.e., smaller standard 
errors) in the estimates for high-poverty districts, but oversampling reduces precision (i.e., increases the 
standard errors) for estimates about districts nationwide. 

Oversampling was accomplished by multiplying the student enrollment for the high-poverty 
districts by a factor of roughly 2.75. This increased their probability of selection in the probability 
proportionate to size (PPS) approach described below relative to districts of the same enrollment in the 
complement stratum. 

Districts with one school (e.g., these can be charter schools that also are a district) were 
undersampled to minimize the numbers of these districts in the final sample, while still representing 
them in the nationally representative sample. Undersampling was accomplished by multiplying the 
student enrollment for these districts by a factor of 0.25. 

Districts were selected using a PPS sampling approach. A district's probability of selection was 
based on its measure of size (which is its student enrollment multiplied by over- and/or undersampling 
factors) relative to a sampling interval. A sampling interval for a stratum is based on the aggregated 
measure of size (based on the frame) divided by the expected district sample size. 


Low and Medium Poverty Stratum includes all districts with less than 21.6 percent of children in poverty according to 2008 
Small Area Income and Poverty Estimates (SAIPE) data from the U.S. Census Bureau. Seventy-five percent of students were 
enrolled in districts with poverty rates below this poverty cut point. High Poverty Stratum includes all districts with poverty 
rates over 21.6 percent. Twenty-five percent of students were enrolled in districts over this poverty cut point. The high- 
poverty districts were oversampled (districts in highest quartile for poverty), as was also done in the NLS-NCLB study. The 
oversampling made the sample percentage of high poverty districts in this study slightly less than 50 percent, while in the 
NLS-NCLB study it was slightly larger than 50 percent. A total of 13,491 districts were found directly on the SAIPE district-level 
database. Non-SAIPE districts were linked by zip code or city name to their matching primary school district, and were 
assigned the a poverty estimate from that primary school district with the same geographic location. In a few cases, a match 
could not be made by zip code or city name, and these had imputations for percentage of children in poverty generated using 
percentage of students eligible for free or reduced lunch when that was available (the correlation between percentage of 
children in poverty and percentage of students eligible for free or reduced lunch was above 75 percent in districts which had 
both values present). There were a few final small specialized districts which could not be assigned poverty percentages by 
either matching or by using percentage of students eligible for free or reduced lunch (as this was missing), and were 
ultimately assigned to the low-poverty stratum. 

79 The RTT Winners are MA, Rl, NY, MD, DC, DE, TN, OH, NC, GA, FL, HI. The RTT Finalists are NJ, PA, KY, SC, IL, CO, LA, AZ, CA. 
The remainder stratum consists of all other states. 
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Sampling intervals and districts selected with certainty (i.e., those districts with a 100 percent 
chance of selection) were identified in an iterative process. First, a sampling interval was computed. 
Next, any districts with measures of size larger than the sampling interval were set aside as certainties 
for the sample. Following this set-aside, we recomputed the sampling interval with the districts that 
were left (their aggregated measure of size divided by the sample size after subtracting off the certainty 
count). As this recomputed sampling interval is always smaller, this generates a new set of districts 
whose measures of size exceeds the new sampling interval, and these were then also designated 
certainties and were set aside. This methodology is standard in many nationally representative 
educational samples (such as the National Assessment of Educational Progress). 

The process of computing sampling intervals and assigning certainties was continued in an 
iterative process until a recomputed sampling interval does not yield new certainties. That is, there are 
no districts whose measure of size exceeds the sampling interval. This sampling interval then becomes 
the final sampling interval because all remaining districts have enrollments less than the interval. These 
remaining districts are the noncertainty districts. Three hundred and seventy-eight large districts with a 
measure of size greater than the final sampling intervals were set aside as certainties after this iterative 
process. 

A sample was drawn of 1,322 noncertainty districts from the frame using the final sampling 
intervals in each district stratum. Within each stratum, we sorted districts into an ordered list based on 
their one-school/charter status 80 , FY2010 Teacher Incentive Fund (TIF) grant recipient status, 
urbancentric locale code, and enrollment; and sampled systematically using this sorted list. 81 This allows 
for some control of variation for these characteristics. We present in table A-l the final sample sizes 
(certainties and noncertainties) for the primary district strata. 


District has more than one school; district has one charter school; district has one regular school. 

81 The FY2010 TIF grantee awards are found at: http://www2.ed.gov/programs/teacherincentive/apps/index.html . Urbanicity 
classifications are central city, urban fringe, town, and rural. 
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Table A-l. Final district sample sizes by district poverty stratum, state's RTT stratum, and certainty 
status 


District 

poverty 

stratum 

RTT status for 
the district's 
state 

Frame 

enrollment 

(number) 

Frame 

enrollment 

(percent) 

Cer- 

tainty 

districts 

Non- 

certainty 

frame 

size 

Non- 

certainty 

sample 

size 

District 

sample 

size 

High 

1-RTT Winner 

3,389,448 

6.94% 

58 

804 

126 

184 

High 

2- RTT Finalist 

3- non RTT 

4,212,656 

8.62% 

105 

1,002 

156 

261 

High 

state 

4,590,248 

9.40% 

94 

2,144 

272 

366 

High 

Total 

12,192,352 

24.96% 

257 

3,950 

554 

811 

Low 

1-RTT Winner 

10,202,655 

20.88% 

47 

1,997 

165 

212 

Low 

2-RTT Finalist 

11,151,200 

22.82% 

33 

3,217 

257 

290 

Low 

3-non RTT 

15,309,843 

31.34% 

41 

6,090 

346 

387 

Low 

Total 

36,663,698 

75.04% 

121 

11,304 

768 

889 

Total 

Total 

48,856,050 

100.00% 

378 

15,254 

1,322 

1,700 


School Sample Design 

We drew a sample of 3,800 schools within the 1,700 sampled districts, with a goal of selecting at 
least two schools per sampled district. The school frame was constructed using school information from 
the 2008-09 CCD with preliminary updates for the 2009-10 school year for sampled districts. The frame 
excluded schools with no enrollment, schools which had only pre-school through kindergarten grades, 
as well as schools that served only special needs populations (e.g., special education only schools). The 
school frame included 38,928 schools within the sampled districts with an enrollment of 25,456,707 
students. 

Schools in sampled districts with only one or two schools (N=192) were selected with certainty 
into the sample. As with the district sample, we used a PPS sampling approach, where sampling intervals 
and school certainties were identified in an iterative process. The probability of selection was 
proportional to the measure of size (which equals school enrollment in general, with an extra 
oversampling factor for persistently lowest-achieving (PLA) schools 82 ) divided by the district probability 
of selection. 83 PLA schools were oversampled to achieve an expected sample size of 570. This allows for 


82 Persistently lowest-achieving schools means, as determined by the State: (a) Any Title I school in improvement, corrective 
action, or restructuring that — (i) Is among the lowest-achieving five percent of Title I schools in improvement, corrective 
action, or restructuring or the lowest-achieving five Title I schools in improvement, corrective action, or restructuring in the 
State, whichever number of schools is greater; or (ii) Is a high school that has had a graduation rate that is less than 60 
percent over a number of years; and (b) Any secondary school that is eligible for, but does not receive. Title I funds that — (i) Is 
among the lowest-achieving five percent of secondary schools or the lowest achieving five secondary schools in the State that 
are eligible for, but do not receive. Title I funds, whichever number of schools is greater; or (ii) Is a high school that has had a 
graduation rate that is less than 60 percent over a number of years. (U.S. Department of Education, November 2010, 

Guidance on FY2010 School Improvement Grants, p. 1. downloaded from: 

http://www2.ed.gov/programs/sif/sigguidancell012010.pdf ). School PLA designations are from approved state applications 
for School Improvement grants for school performance status. Retrieved December 2010 from 
http://www2.ed.gov/programs/sif/ 

83 This is generally done in nested multistage probability sampling, where units at one level (schools) are drawn within the 
sampled units from a higher level (districts). Once the district is sampled, all of its nested schools receive boosts in their 
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more precision (smaller standard errors) in the estimates for these schools. Oversampling was 
accomplished by multiplying the school enrollment for PLA schools by a factor of 8. 

In processing the school frame using the school's adjusted measure of size (adjusted by dividing 
by the district probability of selection), an additional 230 schools were identified as certainties. These 
were schools with adjusted probabilities of selection (after division by the district probability selection) 
greater than 1. 

The remaining 3,378 schools to sample from a frame of 38,506 schools were selected using a 
statistical method called "balanced sampling," which controlled the number of schools sampled for each 
sampled district 84 and balanced across two other stratification levels (or dimensions, described below). 

The school frame of 38,506 schools was divided into eight major strata 85 based on high 
poverty/low poverty status of the district, certainty or noncertainty status of the district, and whether or 
not the district had at least one persistently lowest-achieving (PLA) school. Across the entire national 
school sample, we balanced on the following two dimensions nested within eight primary district strata: 

• Dimension 1: school level/span (elementary, high, middle and other) 86 by school performance 

status (PLA; not PLA, but the school is in need of improvement (SINI) 87 ; or other). 

• Dimension 2: school level/span by school size (large/small) for elementary and high schools. 

Middle and other was not separated out by school size. 88 

All 38,506 schools had assigned probabilities of selection strictly less than 1. Their probabilities were 
raked 89 to add to the desired district sample sizes for each raking dimension cell. One raking dimension 


measure of size (through dividing by the district probability of selection) to make sure the school sample is appropriately large 
enough within the sampled district. For example, if the district probability of selection is 20 percent and the district is 
sampled, then the schools in that district have their measures of size multiplied by 5. In a sense, the sampled schools in that 
district are representing all schools in that district, and also the schools in four other districts which were not sampled (the 
district was sampled at a rate of 1 in 5). Hence we want a five-times bigger school sample in that district. 

84 Schools in districts with only one or two schools were selected with certainty. The school sample size for remaining districts 
was proportional to the relative aggregated measure of size for the district, as is standard in multistage probability sampling. 
1,509 districts had a school sample size of 2. Eighty-four larger districts had a school sample size of 3 or larger because of their 
larger aggregate measures of size. The largest school sample size for a district was 77 schools. 

85 The high poverty/low poverty district strata and the PLA inclusion strata were added as major strata as these were contained 
units designated for oversampling, and the stratification helped to control this. The certainty/noncertainty district strata were 
added as major strata to facilitate variance estimation. Variance estimation is based on the highest stage of sampling, which is 
at the district level for noncertainty districts and at the school level for certainty districts (for certainty districts there is no 
actual sampling at the district level). 

86 Elementary is defined to have a low grade below fourth grade, and a high grade no higher than eighth grade. Middle is 
defined to have a low grade of fourth through seventh, and a high grade no higher than ninth grade. High is defined to have a 
low grade no lower than seventh, and a high grade of twelfth. Other schools are all other schools. 

87 Under the No Child Left Behind Act of 2001, schools that have not met state established student achievement goals for two 
or more years are identified as schools in need of improvement (SINI). The schools designated as "not PLA, but SINI" used for 
sampling are the Tier III schools identified in the approved state applications for School Improvement grants for school 
performance status. Retrieved December 2010 from http://www2.ed.gov/programs/sif/. 

88 For elementary/high schools, "large" is defined to be larger than the median enrollment for all elementary/high schools in 
the US. For middle and other schools, there was no separation into large and small in the stratification. 


A- 8 


is the district, one is school span/school size, and one is school span/school performance status. For 
example, for the district dimension, most desired district sample sizes were 2 or 3 schools. 

School sampling was performed separately for each of the eight major strata. The object of this 
balanced sampling was to select the schools respecting the final measures of size within each major 
district stratum while simultaneously respecting three sets of stratification sample sizes (i.e., it matches 
(or respects) the multiple constraints as closely as possible). We present in table A-2 the school frame 
and school sample sizes by the major district strata. 

Table A-2. School frame and sample sizes for the major district strata 


District 

poverty 

stratum 

District 

certainty 

stratum 

District 

PLA 

status 1 

School 
frame count 
(excludes 
initial 
certainties) 2 

School 

sample 

size 

targets 3 

School 
certainties 
identified 
by PPS 
approach 

School frame 
count 
(excludes all 
certainties) 

School 
sample size 
(excludes all 
certainties) 4 

High 

Noncrt 

noPLA 

2,700 

796 

15 

2,685 

781 

High 

Noncrt 

w/PLA 

819 

191 

77 

742 

114 

High 

Cert 

noPLA 

2,782 

269 

0 

2,782 

269 

High 

Cert 

w/PLA 

9,713 

621 

93 

9,620 

528 

Low 

Noncrt 

noPLA 

8,825 

1,270 

10 

8,815 

1,260 

Low 

Noncrt 

w/PLA 

2,557 

179 

35 

2,522 

144 

Low 

Cert 

noPLA 

4,518 

130 

0 

4,518 

130 

Low 

Cert 

w/PLA 

6,822 

152 

0 

6,822 

152 

All 

All 

All 

38,736 

3,608 

230 

38,506 

3,378 


No PLA indicates that the district did not have any PLA schools. W/PLA indicates that the district had at least one PLA school. 

2 The school frame count excludes the 192 schools identified as initial certainty schools because they are in a district with only 

one or two schools. 

3 The school sample size is the targeted school sample size by district strata with the initial set of 192 certainty schools 

removed. 

4 These 3,378 schools were selected through the balanced sampling procedure. 

The desire to balance simultaneously on three stratification structures required the study team 
to use special methods, based on the work of Deville and Ti lie (2004). We found, however, that the 
software available for doing this was inadequate; it did not allow us to balance completely at the district 
level, guaranteeing at least two sampled schools per district (which was a study requirement) because of 
the large sample sizes. We developed an algorithm for executing balanced sampling using Westat's 
sampling and calibration software. 90 

At the conclusion of the balanced sampling process, the assigned school sample sizes for 
districts were matched exactly, and assigned school sample sizes for the grade span-performance strata 


Raking is also called iterative proportional fitting (IPF). IPF adjusts probabilities (or weights, etc.) to aggregate to certain 
desired cell totals, while being as close as possible to the original values. (The cells are generally marginal cells for a multiway 
table.) 

90 We deviated from the Deville and Tille approach as is implemented in the software macro samplecube from https://cran.r- 
proiect.org/web/packages/sampling/sampling.pdf , as we found that the level of achieved balance was not adequate for the 
evaluation's purposes. The approach we used in place utilizes Westat production software macros WESSAMP and FSRAKE in a 
sequential process. The former macro draws stratified probability proportionate to size systematic samples, and the latter 
macro carries out iterative proportional fitting (raking). 


A- 9 


(elementary, middle, high, other crossed with PLA, Other SINI, Non SINI) and span-size strata 
(elementary: large and small, middle and other, high: large and small) matched not exactly, but with 
very small deviations from balance. 91 In the final drawn sample, 543 PLA schools were sampled. 


Response Rates and Weighting 


Administration for the spring 2011 surveys started in April and continued through July for the 
SEA survey. The majority of responding districts and school submitted their surveys by the end of August 
2011, but data collection was kept open into November to accommodate the schedules of respondents 
for particular districts and schools. The study team reminded participants completing the survey after 
the 2010-11 school year that they should think back to the previous year when answering the survey. 
Administration for the spring 2012 surveys started in May 2012 and continued into August 2012 for the 
SEA survey. The majority of responding districts and schools submitted their surveys by the end of 
August 2012. However, data collection was kept open into October 2012 to accommodate respondents 
in a few districts and schools. These respondents were reminded that they should think back to the 
2011-12 school year when completing the surveys. 

All SEAs responded to both surveys. For the 2011 data collection, 88 percent of districts 
(unweighted) and 78 percent of schools responded to the web surveys. The response rates increased to 
91 percent for districts, but decreased to 72 percent for schools in the 2012 data collection. 

The state-level survey data requires no weights, as every state was canvassed in this study. 
District and school survey data were weighted to generate the cross-sectional estimates found in this 
report. We generated final cross-sectional weights for each survey year: 2009-10, 2010-11, and 2011-12. 
However, given that the data for the 2009-10 and 2010-11 school years came from the same 
respondents to the 2011 survey administration, each respondent's final weight is the same for those 
school years. 

Although the district and school sample designs used a probability proportionate to size (PPS) 
approach (where larger districts and schools had a higher probability of selection than they would have 
had otherwise), the district and school percentages in this report were generated using "unit-based" 
weights. The unit-based weights are appropriate when generating estimates to answer the research 
questions for this report which ask about the percentages of districts and schools implementing reforms 
and reporting major challenges. However, these weights do have higher variability than "enrollment- 
based" weights (i.e., where the base weight is equal to the district's/school's enrollment multiplied by 
the inverse of the district's/school's probability of selection.) 92 The enrollment based weight starts with 
the base weight E/pi, where E is the school enrollment, and pi is the school probability of selection. The 
unit-based weight starts with 1/pi. Pi is proportional itself to a function of enrollment, which varies 


91 The differences between the final realized sample size and the target sample size for the span-performance strata and the 
span-size strata were small (by only one or two schools, with a few cases of up to five schools [INSERT ) ], but the difference 
was not systematic in any particular direction. 

92 The study team also generated a set of "enrollment-based'' weights. The enrollment-based weights can be used to make 
unbiased estimates of total enrollment in districts or schools nationwide. The team also generated a set of unit-based and 
enrollment-based longitudinal weights, which can be used with the subset of districts and schools that responded to both the 
2011 and 2012 surveys. The enrollment-based and longitudinal weights were not used for the analyses in this report but are 
available to users through the restricted-use data file available through the IES Data Security Office to licensed users. 
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considerably across districts (and somewhat less across schools), so the unit-base weight varies 
considerably. But E/pi does not vary much, as pi is close to proportional to E.). All other things being 
equal, a higher variability in the weights leads to higher standard errors. (See for example, Valliant, 
Dever, Kreuter, 2013, Section 14.4.) However, at the school level, school enrollment sizes are less 
variable than the district enrollment sizes, so the design effects from having unit-based estimates with a 
PPS sample are smaller for schools than for districts. 93 

The following describes the process of developing the final unit-based sampling and replicate 
weights for the district and school data from the 2011 and 2012 surveys. 

District Weights 

Base weights 

We first generated base sampling weights. The base sampling weight for the unit-based district 
weight is equal to the inverse of the district probability of selection. The unit-based weights when 
aggregated generate unbiased estimates of total districts 94 (this is the well-known 'Horvitz-Thompson 
estimator': see for example Valliant et al., 2013, Section 13.3) 

Replicate weights 

While we can use final sampling weights alone to obtain approximately unbiased percentage 
estimates, we needed to apply appropriate variance estimation techniques to produce approximately 
unbiased estimates of the standard errors (Brick, Morganstein, and Valliant, 2000, p. 2). As a result, we 
relied on replication methods 95 and generated cross-sectional district replicate weights for each survey 
year. For districts selected with certainty into the sample, the replicate weights are equal to the base 
sampling weights. For the noncertainty districts, the replicate weights were generated using the JK2 
jackknife replication method (the JK2 method is discussed in for example Valliant et al.,2013, p. 420), 
with the variance strata being based on the ordering of districts on the district frame, nested within the 
primary district strata. Appropriate finite population corrections were incorporated into the replicate 
weights (see for example Rizzo and Rust, 2011). 

Nonresponse and benchmarking adjustments 

We incorporated nonresponse adjustments to the sampling and replicate weights since the 
district response rate was not 100 percent. The adjustments for the cross-sectional weights for the 2012 
survey data was done in a similar way as for the 2011 survey data, except that the nonresponse 
indicators were updated to indicate a differing set of respondents (if some schools responded for one 
year, but not the other). We applied the same approach for nonresponse and benchmarking 
adjustments to the base sampling and replicate weights from both survey administrations. The 
adjustments were done according to a raking process, in which district base and replicate weights for 


Note that the previous NLS-NCLB study also generated both enrollment-based and unit-based weights. 

94 This is true for any district subgroup. 

95 As noted in Brick et al. (2000), replication involves repeatedly selecting subsamples from the full sample. The desired 
statistics are computed from each subsample, and the variability among these subsamples or replicate estimates is used to 
compute the standard error of the full sample estimate (pp. 2-3). 
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responding districts were calibrated to make sure the totals match unit totals for a variety of district 
characteristics 96 . The raking adjustments simultaneously adjust for nonresponse and improve precision. 
Note that this adjustment approach was similar to that used in other U.S. Department of Education 
studies of district policy implementation. 97 

Raking adjustments were nested within the four cells defined by district certainty status and 
high/low poverty status. Within these four cells the raking dimensions are RTT Winner, RTT Finalist, non 
RTT state crossed with Census region (a total of 12 cells); and Urbanicity (1— Central City; 2— Urban 
Fringe; 3— Town; 4— Rural) (a total of 4 cells). The control totals for the unit-based weights are district 
totals for each of these cells using the final district frame. These adjustments make sure the weighted 
percentages at the sample level were exactly equal to the exact percentages at the frame level for these 
important district subgroups. This completely re-balances the responding district sample to the frame 
for these cells (and also eliminates any bias that would arise from an imbalance due to higher or lower 
mean response for these cells). The raking approach is based on the presupposition that response 
propensity (the underlying propensity to respond for any sample unit) fits a main-effects model with 
predictor variables being the cells from the two raking dimensions. We checked for deviations from this 
main-effects model (i.e., significant interaction terms) using logistic regression and a data mining tool. 
We did not find these interactions: the main-effects model was found to fit well. 

School weights 

Base weights 

For the schools, the base sampling weight for the unit-based weight is equal to the inverse of 
the school's probability of selection, which includes the school's district's probability and the conditional 
school probability given the school's district was selected. As with the district base weight, the unit- 
based base weights when aggregated generate unbiased estimates of total schools 98 . 

Replicate weights 

Within the noncertainty districts, district selection is the first stage of selection. Schools are the 
second stage of selection. For variance estimation purposes, the key level to capture is always the first 
stage of selection. In this case, it is district selection: the replicate weights should primarily reflect non- 
certainty district sampling. The variance from district sampling is correctly measured by the district 
replicate weights, which are then multiplied by the conditional school selection factor (the inverse of the 
conditional school probability of selection) to get to the school level. But the finite population 


Doing this will improve the precision for any characteristics which are correlated to the characteristics defining the raking 
cells. 

97 For the NLS-NCLB and this study, nonresponse adjustments and calibration (raking) adjustments were made to the weights. 
The district-level response rate for NLS-NCLB was 96 percent in 2004-05 and 99 percent in 2006-07. The district-level response 
rates for this study were 88 percent for 2011 and 91 percent for 2012. Calibration in NLS-NCLB was for district size (four cells), 
district poverty status (high and other), Census region, and metropolitan status (two cells). Calibration for this study was for 
district poverty status (high and other), RTT status (RTT Winner, RTT Finalist, Other), Census Region, and urbanicity (four cells). 
Calibration adjusts the weights to better correspond to the proportions of the population in each of these categories. NLS- 
NCLB did a limited amount of trimming of extreme weights that was not done in this study (trading off in NLS-NCLB a small 
amount of potential bias for a slight reduction in variance). 

98 This is true for any school subgroup. 
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corrections" at the school level are not properly reflected by these adjusted district replicate weights, 
so we need an extra set of school-specific replicate weights to reflect properly the school finite 
population correction (which is not equal to the district finite population correction). This methodology 
of creating school-level replicate weights also was used in recent (2011 and after) National Assessment 
of Educational Progress studies (as described in Rizzo and Rust, 2011). 

Within the certainty districts, school selection is the first stage of selection. The schools were 
sampled using the district as strata, but also utilizing a balanced sample design in order to balance to 
span-performance and span-size strata as well. As a result, the replicate weights for schools in these 
districts are based on the Fay's Method replication approach. 100 

Nonresponse and benchmarking adjustments 

Given that the school response rates for both administrations were less than 100 percent, we 
implemented nonresponse adjustments to the base sampling and replicate weights. The adjustments for 
the cross-sectional weights for the 2012 survey data was done in a similar way as for the 2011 survey 
data, except that the nonresponse indicators were updated to indicate a differing set of respondents (if 
some schools responded for one year, but not the other). 

These adjustments for the 2011 data were according to a raking process, which raked the school 
base weights (and school replicate base weights) for the responding schools to control totals for cells in 
several dimensions. The original school frame (before subsetting to the sampled districts) was utilized to 
provide these school count control totals within each. 

There were three raking dimensions. The first two dimensions were determined by school 
stratification (defined below), and the third dimension was determined by the results of a weighted 
logistic regression analysis. For the weighted logistic regression analyses, the dependent variable was 
school response/nonresponse. The weight was the unit-based school base weight (the inverse of the 
school's overall probability of selection). The predictor variables included the school's Race to the Top 
Stratum (RTT award state, RTT Finalist state, all other states), Census Region or Census Division, poverty 
status (two cells: high poverty and low/medium poverty, or three cells: high poverty, medium poverty, 
low poverty), and urbanicity with or without crossing with city size (four cells or six cells respectively). 
The weighted logistic regression analyses were used to identify a third dimension of response 
adjustment cells. This analysis resulted in the selection of raking dimensions for the raking nonresponse 
adjustments. 


99 The finite population correction is the reduction in variance that results from a nonnegligible sampling fraction. For example, 
if the sampling fraction is 20 percent, the finite population correction is 80 percent. This reduction in variance reflects that 20 
percent of the population is actually known. See for example Cochran (1977), Section 2.6. 

100 Deville and Tille 2005 provide approximate estimators for the balanced sampling design. Kim and Wu (2011) recommend 
Fay's Method for estimating Deville and Tille's approximate variance. 
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The raking procedures were done separately for the four cells based on district certainty status 
and poverty status (high poverty vs. low/medium poverty). The three dimensions for unit-based weights 
within the four primary cells are as follows 101 : 

(1) School performance strata (PLA 102 , nonPLA SINI 103 , and nonSINI) nested within school 
span strata (elementary, middle, high, combined); 

(2) School size strata (small, medium, large) nested with school span strata; 

(3) RTT stratum crossed with Census region crossed with urbanicity (four cells). 

The raking procedure brings the nonresponse-adjusted weights for the responding school set 
within each raking cell back to the Common Core of Data (CCD) frame totals for the cells 104 , completely 
re-balancing the responding school sample to the frame for these cells (and eliminating any bias that 
would arise from an imbalance due to higher or lower mean response for these cells). This was done 
successfully, at least along these dimensions 105 , the effects of school nonresponse was fully adjusted for 
the weights for the 2011 survey data. 

Generalizability of District and School Estimates 

The estimates from the district and school samples in this report generalize to 2009-10, 2010- 
11, and 2011-12 activity for districts and schools in operation during the 2010-11 school year . That is, 
the weighted cross-sectional 2009-10 estimates represent what 2010-11 districts were doing in 2009- 
10. The weighted cross-sectional 2011-12 estimates represent what 2010-11 districts were doing in 
2011-12 (among those that were still operating in 2011-12). 

The samples cannot be used to generalize to the 2009-10 population of school districts because 
we classified as ineligible for the study those districts and schools that were operating in 2009-10, but 
not operating in 2010-11. The samples cannot be used to generalize to the 2011-12 population of school 
districts and schools because we did not refresh the samples to try to incorporate those districts and 
their schools that started operating in 2011-12. 


Analysis 


This section begins with an overview of the indicator approach to measure reform 
implementation, and then provides more details about the construction of the indicators. The details 
include the treatment of state activities that may continue beyond one year and the treatment of item- 
level missing data in the construction of the district and school indicators. See appendix B for 
information specific to each indicator, including its component items and rules for using component 


101 Some collapsing of these cells was done if the cell sizes were too small (less than 10) or the cell adjustments too extreme. 

102 Persistently Lowest-Achieving school. 

103 School in Need of Improvement. 

104 It should be noted that we did find some differences in response rates across these cells. 

105 Nonresponse bias will also be reduced for any characteristics correlated in the population to these cell categories, with the 
degree of reduction roughly proportional to the degree of correlation with these cell categories. 
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items to create the indicator (e.g., whether an indicator is met if district responds 'yes' to at least one of 
the items or all of the items). 

Overview of Indicator Approach 

To address the study questions, we developed indicators of reform implementation at the state, 
district, and school levels. The indicators reflect ED's priorities and key reform strategies within each of 
the four assurance areas. The indicators were designed to provide a high-level snapshot of whether 
SEAs, districts, or schools had a particular policy in place, provided support, or carried out a particular 
activity. The indicators do not describe the quality or intensity of reform implementation. For most 
indicators, an SEA, district, or school was said to have met an indicator if it reported implementing any 
one of a particular set of related strategies. For a smaller set of indicators, an SEA or district met the 
indicator only if it met more than one requirement. 

Because of the variety of potential SEA and district responses to Recovery Act reform 
requirements and because assurances could be met by using different approaches, the indicators often 
captured several ways in which a state, district, or school might implement a reform. For example, SEAs 
could use multiple strategies in their role to improve standards and assessment, from providing 
professional development directly to supporting LEA professional development through guidance and 
technical assistance. Similarly, districts could use multiple strategies to support or promote the 
implementation of new standards and assessments in schools, from distributing instructional materials 
to providing criteria for schools to use when selecting new aligned curriculum. Where appropriate, the 
indicators include multiple strategies, but do not assume that one approach is preferable to another. 
Some Recovery Act programs, however, have more prescriptive requirements. In these cases, states and 
districts had to take specified actions, such as adoption of the CCSS or the inclusion of student growth 
measures in educator evaluation systems, to meet an indicator. 

Calculating SEA Indicators 

All but three of the 18 SEA indicators of reform are based on the 2011 and 2012 SEA surveys. In 
these surveys, SEAs were asked to check a box (see exhibit A-l)if they had a particular role in reform 
implementation (e.g., simplifying or shortening the process of obtaining full licensure ); if their states' 
evaluation, compensation, or data system included specific elements (e.g., their teacher evaluation 
system used rating scales or rubrics that define three or more levels to evaluate classroom instruction or 
practice); or if they provided supports to LEAs implementing reform (e.g., funding, technical assistance, 
training). In the analysis of these data, an SEA was classified as having implemented a reform if they 
checked the relevant box. The responses to related survey items were combined following indicator- 
specific decision rules to determine whether the SEA met the indicator. For example, to meet the 
requirements of the indicator SEA Facilitated Educators' Access to Assessment Data, an SEA had to 
either (1) check the box indicating they facilitated access to new or existing assessment data by 
providing educators with key LEA, school and student indicators through report cards, data dashboards, 
or other feedback and analysis systems or (2) check the box indicating they facilitated access to new or 
existing assessment data by establishing and maintaining state data systems that share longitudinal data 
on students with local data systems. See appendix B for the requirements for all indicators. 
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Exhibit A-l. Example of SEA survey response format 


Components Included in Statewide System or Standards, Guidelines 
and/or Technical Assistance forTeacher Evaluation Systems 

Included in 
2009-2010 
(Check all that 
apply.) 

Included in 
2010-2011 
(Check all that 
apply.) 

A rating scale or rubric that defines three or more performance levels to 
evaluate classroom instruction or practice 

□ 

□ 


In developing the SEA indicators, the study team recognized that some reform activities have 
the potential to continue over multiple years once they are instituted. For example, when a state issues 
standards or guidelines for teacher preparation programs, these are likely to remain in place until new 
or revised standards or guidelines are issued. So, an action in one year can be viewed as ongoing or 
providing continued support for a reform effort in subsequent years. In measuring SEA reform activity 
for selected indicators, if a state reported activity in a previous year (i.e., 2009-10 or 2010-11), the 
analyses counted the SEA as meeting the indicator in subsequent years. In general, we treated the 
following types of SEA activities as actions that can represent ongoing or continued support in future 
years: 


■ Provided state-developed professional development to LEAs, 

■ Supported LEA-designed professional development by providing funding, guidance and 
technical assistance, or "train the trainers" sessions, 

■ Identified, developed, and/or distributed instructional materials to LEAs, 

■ Simplified or shortened process of obtaining teacher licensure/certification, 

■ Authorized independent providers to provide teacher training, 

■ Issued standards or guidelines, and 

■ Passed legislation to increase the permissible number of charter schools or removed 
prohibitions on charter schools. 

Twelve of the 18 state indicators included activities treated in indicator construction as ongoing 
beyond the initial year that activity was reported. In table A-3, we present each SEA indicator, identify 
those that consider activity in prior years as ongoing into the current year, and include the number of 
SEAs meeting an indicator in a given year based solely on the survey responses for that year and the 
number meeting that indicator based on on-going type activities reported for a prior year. 
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Table A-3. SEA indicators' consideration of ongoing activity and the number of SEAs meeting each indicator through the current year's 
requirements or through previous years' activities, by assurance area and school year 


Indicator 
considers activity 
from previous 


Indicator by assurance area 

year 

2009-10 

2010-11 

2011-12 

Standards and assessments 

State had adopted the Common Core State Standards in 
mathematics and reading/English language arts 

No 


In current year [43] 
(2010-11) 

In current year [46] 
(2011-12) 

SEA provided, guided or funded professional development 
on the Common Core State Standards 

Yes 


In current year [37] 
(2010-11) 

In current year [44] 
or prior year [1] 
(2011-12 or 2010- 
11) 

SEA provided instructional materials or curriculum 
assistance for the Common Core State Standards 

Yes 

Not applicable 

In current year [29] 
(2010-11) 

In current year [39] 
or prior year [3] 
(2011-12 or 2010- 
11) 

State was a member of a federally funded consortium 
developing assessments aligned to the Common Core State 
Standards 

No 


In current year [43] 
(2010-11) 

In current year [44] 
(2011-12) 


continued 
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Table A-3. SEA indicators' consideration of ongoing activity and the number of SEAs meeting each indicator through the current year's 
requirements or through previous years' activities, by assurance area and school year (cont'd) 


Indicator by assurance area 

Indicator 
considers 
activity from 
previous year 

2009-10 

2010-11 

2011-12 

Data systems 

State operated a longitudinal data system that included 12 
core components 

No 

In current year [13] 
(2009-10) 

In current year [13] 
(2010-11) 

Not available 

State data system had ability to link teachers to student 
data 

No 

In current year [30] 
(2009-10) 

In current year [30] 
(2010-11) 

SEA facilitated educators' access to assessment data 

No 

In current year [35] 
(2009-10) 

In current year [43] 
(2010-11) 

In current year [45] 
(2011-12) 

SEA provided professional development or technical 
assistance to support educators' use of assessment data 

Yes 

In current year [32] 
(2009-10) 

In current year [40] 
or prior year [0] 
(2010-11 or 2009- 
10) 

In current year [43] 
or either of two 
prior years [4] 
(2011-12 or 2010-11 
or 2009-10) 


continued 
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Table A-3. SEA indicators' consideration of ongoing activity and the number of SEAs meeting each indicator through the current year's 
requirements or through previous years' activities, by assurance area and school year (cont'd) 


Indicator by assurance area 

Indicator 
considers 
activity from 
previous year 

2009-10 

2010-11 

2011-12 

Educator workforce development 

SEA simplified or shortened educator licensure process 
or authorized non-university preparation programs 

Yes 

In current year [33] 
(2009-10) 

In current year [35] 
or prior year [0] 
(2010-11 or 2009-10) 

In current year [33] or either 
of two prior years [6] (2011- 
12 or 2010-11 or 2009-10) 

SEA issued standards or guidelines for teacher 
preparation programs 

Yes 

In current year [28] 
(2009-10) 

In current year [32] 
or prior year [0] 
(2010-11 or 2009-10) 

In current year [26] or either 
of two prior years [12] 
(2011-12 or 2010-11 or 
2009-10) 

SEA issued standards or guidelines for principal 
preparation programs 

Yes 

In current year [20] 
(2009-10) 

In current year [25] 
or prior year [1] 
(2010-11 or 2009-10) 

In current year [24] or either 
of two prior years [11] 
(2011-12 or 2010-11 or 
2009-10) 

SEA supported use of multi-level ratings, multiple 
observations, and student achievement gains for 
teacher evaluation 

Yes 

In current year [1] 
(2009-10) 

In current year [2] or, 
if sub-set of 
conditions were met, 
prior year [0] (2010- 
11 or conditionally 
2009-10) 1 

In current year [14] or, if 
sub-set of conditions were 
met, prior years [0](2011-12 
or conditionally 2010-11 or 
2009-10) 

SEA supported use of student achievement gains for 
principal evaluation 

Yes 

In current year [6] 
(2009-10) 

In current year [9] or, 
if sub-set of 
conditions were met, 
prior year [0] (2010- 
11 or conditionally 
2009-10) 

In current year [21] or, if 
sub-set of conditions were 
met, prior years [1] (2011-12 
or conditionally 2010-11 or 
2009-10) 


continued 
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Table A-3. SEA indicators' consideration of ongoing activity and the number of SEAs meeting each indicator through the current year's 
requirements or through previous years' activities, by assurance area and school year (cont'd) 


Indicator by assurance area 

Indicator 
considers 
activity from 
previous 
year 

2009-10 

2010-11 

2011-12 

SEA supported differentiating teacher compensation based on 
student achievement gains 

Yes 

In current year [7] 
(2009-10) 

In current year [9] or, 
if sub-set of 
conditions were met, 
prior year [0] (2010- 
11 or conditionally 
2009-10) 

In current year [13] 
or, if sub-set of 
conditions were 
met, prior years [2] 
(2011-12 or 
conditionally 2010- 
11 or 2009-10) 

Support for improving low-performing schools 

State allowed for expansion of the number of charter schools 

Yes 

In current year [19] 
(2009-10) 

In current year [25] 
or, if sub-set of 
conditions were met, 
prior year [3](2010- 
11 or conditionally 
2009-10) 1 

In current year [25] 
or, if sub-set of 
conditions were 
met, prior years [8] 
(2011-12 or 
conditionally 2010- 
11 or 2009-10) 1 

SEA provided guidance on choosing and implementing school 
intervention models defined by ED 

Yes 

In current year [42] 
(2009-10) 

In current year [50] 
or prior year [0] 
(2010-11 or 2009-10) 

In current year [50] 
or either of two 
prior years [1] 
(2011-12 or 2010-11 
or 2009-10) 


continued 
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Table A-3. SEA indicators' consideration of ongoing activity and the number of SEAs meeting each indicator through the current year's 
requirements or through previous years' activities, by assurance area and school year (cont'd) 


Indicator 
considers 
activity from 
previous 


Indicator by assurance area 

year 

2009-10 

2010-11 

2011-12 

SEA monitored deployment of effective educators in low- 
performing schools 

No 

In current year [7] 
(2009-10) 

In current year [9] 
(2010-11) 

In current year [12] 
(2011-12) 

SEA supported using compensation incentives to improve 
staffing at low-performing schools 

Yes 

In current year [8] 
(2009-10) 

In current year [9] or, 
if sub-set of 
conditions were met, 
prior year [0] (2010- 
11 or conditionally 
2009-10) 

In current year [12] 
or, if sub-set of 
conditions were 
met, prior years [2] 
(2011-12 or 
conditionally 2010- 
11 or 2009-10) 


1 For this indicator, to meet the indicator a state had to either: a) have no cap in the current year on the number of charter schools, or b) had to pass legislation in the current 
year or one of the prior years covered by the study to either increase the permissible number of charter schools or remove prohibitions on charter schools. 
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Calculating District and School Indicators 

At the district level, we created 21 indicators of reform implementation from survey data. When 
reporting on activities for the 2010-11 and 2011-12 school years, districts were asked whether they used 
or included the strategy in all schools (or for all educators), some schools (or for some educators), or if 
they were not using the strategy, whether or not they were actively planning its use or (see exhibit 2). 
For those school years, the district was counted as using the reform strategy if they reported that they 
used or included the strategy in all or some schools. For the 2009-10 school year, the response options 
included only Yes and No. For that school year, districts that responded Yes were counted as using the 
strategy. 

Exhibit A-2. Example of district survey response formats 


Components of Educator Performance 
Evaluation System in Your District 

Component in 
the System m 
2009-2010 

(Check Yes or No) 

Status in 2010-2011 for Educators 

(Check one in each row) 

No Current Plani to hclude the 
Component 

Actively Planning Uie or 
Developing toe Component 

Half of Educ a tors 

or Fewer =i 

B' <t> 

<“> 
*> o 

More Than Half of S 1 "! 

Educ a tori but Not All S s 

Educ a tori | s 

£>» 

: ^ 

All Educ ato ri 

Teacher evaluation system 



Uses a ratmg scale or rubric that defines 

1 1 Yes 



three or more performance levels to 

| | ^ 

□ □ 

□ □ □ 

evaluate classroom retraction or practice 

□ No 




At the school level, we created 22 indicators of reform implementation from survey data. For each 
school year, the school survey asked principals to answer Yes or No about use of particular reform 
activities or practices (see exhibit 3). 

Exhibit A-3. Example of school survey response formats 


Performance Evaluation Practices 

Used in 2009-2010 

(Check Yes or No) 

Used in 2010-2011 

(Check Yes or No) 

Teacherevaluation practices 


Uses a rating scale or rubric that defines three or more performance 
levels to evaluate classroom instruction or practice 

1 I Yes Q No 

1 I Yes Q No 


As with the SEA indicators, for the district and school indicators components were combined 
following the indicator's decision rules to determine whether the district or school met an indicator. For 
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example, to meet the district indicator District Provided Compensation Incentives to Improve Staffing at 
Low-Performing Schools, a district with low-performing schools had to count either as (1) providing 
higher starting salaries, add-ons, stipends, or signing bonuses for teachers who move to teach in low- 
performing schools or (2) providing loan forgiveness or tuition support for teachers who move to low- 
performing schools. See appendix B for the requirements for all indicators. 

When calculating the percentage of districts or schools that met an indicator, item-level missing 
data sometimes prevented us from assigning an indicator value to a specific district or school. The 
denominators for the percentage of districts or schools that met an indicator (e.g., supported educators' 
access to assessment data) includes only the number of districts or schools that provided sufficient data 
to determine whether or not they fulfilled the requirements of the indicator. 106 When constructing 
district indicators, between 0.1 and 9 percent of responding districts did not supply sufficient data to 
determine whether district activities met the criteria for the indicator and were therefore excluded from 
the analysis (see table A-4). For school-level indicators, between 2 and 13 percent of responding schools 
did not supply sufficient data to determine whether they met the criteria for the indicator and were 
excluded from the analysis (see table AS). 

Table A-4. Percentage of districts excluded from indicators, by assurance area and year 


Assurance area 

2009-10 

2010-11 

2011-12 

Standards and assessments 

Not applicable 

4 - 9 % 

0.1 -0.6% 

Data systems 

7% 

8 - 12% 

0.4 -0.7% 

Educator workforce development 

0.2-4% 

4 - 5% 

0.2 % 

Support for low-performing schools 

Not applicable 

Not applicable 

0.4 -0.6% 


Table A-5. Percentage of schools excluded from indicators, by assurance area and year 


Assurance area 

2009-10 

2010-11 

2011-12 

Standards and assessments 

Not applicable 

6 - 9% 

3 - 7% 

Data systems 

8% 

7 - 8% 

4 - 6% 

Educator workforce development 

5 - 6% 

4 - 5% 

2 - 3% 

Support for low-performing schools 

5 - 6% 

3 - 6% 

6 - 13% 


Statistical Tests Used 

In this section, we describe the methodology for tests used for this report that test for 
significant differences in the percentages of districts (or schools) meeting indicators or reporting major 
challenges. Statistical testing was not required to examine the SEA data because those data are universe 
data from SEAs in all 50 states and the District of Columbia. 


106 For example, if to meet an indicator a district had to use at least one of four possible strategies for providing teachers with 
access to assessment data, then a district provided sufficient data if it either a) responded that it implemented one of the 
strategies, or b) responded that they did not implement any of the four strategies. If the district failed to provide an answer to 
one of the strategies and did not implement any of the remaining three, then that district did not provide sufficient data to 
determine whether or not it fulfilled the requirements of the indicator. That is, we were unable to determine if the district 
met the indicator based on the responses available. 
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The report includes findings for four types of comparisons: 


• Comparisons of the percentage of districts (or schools) meeting a particular reform indicator 
over time. For example, is the percentage of districts that provided educators with access to 
assessment data in 2011-12 significantly different from the percentage that did so in 2010-11? 

• Subgroup-to-subgroup comparisons of the percentage of districts (or schools) meeting a 
particular reform indicator in 2011-12. For example, is the percentage of low-performing 
schools where a principal was evaluated based in part on student achievement gains in 2011-12 
significantly different from the percentage of other schools with this practice? 

• Indicator-to-indicator comparisons for 2011-12. For example, was the percentage of districts 
that provided educators with access to assessment data in 2011-12 significantly different from 
the percentage of districts that used longitudinal data to track student achievement gains for 
individual teachers in 2011-12? 

• Challenge-to-challenge comparisons for 2011-12. For example, was the percentage of districts 
that reported current assessments are not aligned with the new standards as a major challenge 
significantly different from the percentage of districts that reported concerns or oppositions 
from school staff about additional assessments? 

Each comparison in this report uses the district and school respondents for a given survey 
administration (spring 2011 or 2012) with sufficient data on each reform indicator. 107 The analyses in 
this report are not limited to the districts and schools that responded to both surveys (i.e., the analyses 
do not use the longitudinal sample). 

We conducted three types of statistical tests for the comparisons described above. These tests 
were run using the final unit-based, cross-sectional replicate weights. These replicate weights take into 
account the complex sample design and nonresponse adjustments. They are valid for computing the 
correct cross-sectional standard errors for that year (taking into account the particular sample 
composition for that year), and also account for the covariance brought about from overlapping samples 
across years in an appropriate and valid way. 

Using the replicate weights to generate replicate standard errors provides approximately 
design-unbiased estimates for the variance for any estimate, including subgroup estimates, cross-year 
comparisons, cross-subgroup comparisons, etc. These standard errors based on the replicate weights 
were used to produce the confidence intervals in this report and for conducting the statistical tests used 
for comparisons described above. The methodology for the three statistical tests is as follows: 


107 As a reminder, the same samples of districts and schools were used for the 2011 and 2012 survey administrations. The 2011 
surveys asked about the 2009-10 and 2010-11 school years. The 2012 surveys asked about the 2011-12 school year. 
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Methodology for the year-to-year comparisons of district estimates and comparisons of school 
estimates 

Given that (1) we drew one sample of districts and one sample of schools that were 
subsequently followed and (2) the analyses rely on the cross-sectional data, we cannot use either the 
standard t-test for independent samples or a paired t-test for dependent samples to test whether year- 
to-year changes in the percentage of districts (or schools) are significantly different. 108 Rather than 
calculating the difference in whether an individual district or school met the indicator in one year 
compared to another year (as we would for a paired t-test), we calculated the estimated percentage 
meeting an indicator across all district (or school) respondents. We then examined the weighted 
difference in the estimated percentages at two time points. We calculated weighted differences using 
the replicate weights, computed the standard errors of the differences, and then ran a two-sided test 
with confidence level 95 percent for the difference of estimates. That is, we tested whether the 
difference between the percentage estimate for an indicator in one year and the percentage estimate 
for the same indicator in another year is zero. The results for these statistical tests are presented in the 
chapter figures and in appendix tables for each assurance area. 

Methodology for statistical tests for subgroup-to-subgroup comparisons of district and school 
estimates 

For the subgroup domains that coincided exactly with the sampling strata (e.g., high-poverty 
district versus districts with other poverty levels), we applied replicate weights and used a standard t- 
test for independent samples to determine whether the percentages of districts (or schools) in different 
domains are significantly different. For subgroup domains that do not coincide exactly with the sampling 
strata, a t-test for independent samples is not precise, as we need to account for the covariance 
between the estimates that may exist. For these comparisons, we used the same approach for the year- 
to-year comparisons described above that accounts for non-zero covariance. The results for these 
statistical tests are presented in the chapter figures and in appendix tables for each assurance area. 

Methodology for statistical tests for within year indicator-to indicator (or challenge-to-challenge) 
comparisons of district and school estimates 

Tests of differences were made within years between indicators (and challenges) within 
assurance areas. These comparisons were similar to the across-year estimates in that the two sample 
sets were similar, but slightly different. This difference comes from differing patterns of missingness 
across indicators (or challenges). In some cases, an indicator would be missing for a given sample unit in 
a given year, and in other cases, not. We carried out the same procedure of creating replicate estimates 
of the differences of the indicators (or challenges), generating a standard error from those replicate 
differences, and then ran the test for the difference of estimates. The results for these statistical tests 
are found in the appendix for each assurance area. 


108 Because more units are the same respondents for the 2009-10/2010-11 data and the 2011-12 data compared to the 
situation when two independent samples are drawn (which would share members only at random), the covariances between 
years are likely to be far larger than what is assumed when applying an independent samples t-test. A paired t-test would 
account for the covariance in the dependent sample, but would require limiting the analysis to the longitudinal sample. 
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Appendix B 

Indicators, Components, and 
Recovery Act Program Requirements 
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Table B-l. Standards and assessments indicators, components, and Recovery Act program 
requirements, by education level 


Indicator 

Components 1 

Recovery Act program requirements 

SEA level 

State had adopted 
the Common Core 
State Standards 
(CCSS) in 

mathematics and 
reading/English 
language arts 

Adopted the CCSS in mathematics and 
English language arts 

Source: 2011 SEA survey, items 1 and 3 

RTT selection criteria (B)(1) 

The extent to which the State has 
demonstrated its commitment to adopting a 
common set of high-quality standards, 
evidenced by— 

(i) The State's participation in a consortium of 
States that— 

(a) Is working toward jointly developing and 
adopting a common set of K-12 standards that 
are supported by evidence that they are 
internationally benchmarked and build toward 
college and career readiness by the time of 
high school graduation; and 

(b) Includes a significant number of States; 

SEA provided, guided 
or funded 
professional 
development on the 
Common Core State 
Standards 

Adopted the CCSS in mathematics and 

English language arts and reported at 

least one of the following professional 

development activities: 

• State-developed professional 
development on CCSS to LEAs in- 
person 

• State-developed professional 
development on CCSS to LEAs online 

• Guidance and technical assistance to 
LEAs on the design and 
implementation of professional 
development 

• "Train-the-trainer" sessions to lead 
LEA staff 

• Funding for LEA-designed 
professional development on 
standards 

Source: 2011 SEA survey, items 1 and 5 

RTT selection criterion (B)(1) 

The extent to which the State has 
demonstrated its commitment to adopting a 
common set of high-quality standards, 
evidenced by— 

(i) The State's participation in a consortium of 
States that— 

(a) Is working toward jointly developing and 
adopting a common set of K-12 standards that 
are supported by evidence that they are 
internationally benchmarked and build toward 
college and career readiness by the time of 
high school graduation 

RTT selection criterion (B)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan for supporting a statewide transition to 
and implementation of internationally 
benchmarked K-12 standards that build toward 
college and career readiness by the time of 
high school graduation, and high-quality 
assessments tied to these standards. State or 
LEA activities might, for example, include: ... 
developing or acquiring and delivering high- 
quality professional development to support 
the transition to new standards and 
assessments. 


(continued) 
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Table B-l. Standards and assessments indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 

SEA provided 

Adopted the CCSS in mathematics and 

RTT selection criterion (B)(1) 

instructional 

English language arts and reported at 

The extent to which the State has 

materials or 

least one of the following support 

demonstrated its commitment to adopting a 

curriculum 

activities: 

common set of high-quality standards, 

assistance for the 

• Identified, developed, and/or 

evidenced by— 

Common Core State 

distributed materials (e.g., 

(i) The State's participation in a consortium of 

Standards 

curriculum guides, pacing guides, 

States that— 


textbooks) aligned with the 

(a) Is working toward jointly developing and 


standards that: 

adopting a common set of K-12 standards that 


o LEAs are required to use 

are supported by evidence that they are 


o LEAs may choose to use 

internationally benchmarked and build toward 


• Provided resources or technical 

college and career readiness by the time of 


assistance to help LEAs map 
curriculum taught to new or revised 

high school graduation 


content standards 

RTT selection criterion (B)(3) 

The extent to which the State, in collaboration 


Source: 2011 SEA survey, items 1 and 5 

with its participating LEAs, has a high-quality 
plan for supporting a statewide transition to 
and implementation of internationally 
benchmarked K-12 standards ... State or LEA 
activities might, for example, include: ... 
developing or acquiring, disseminating, and 
implementing high-quality instructional 
materials and assessments (including, for 
example, formative and interim assessments) ... 
and engaging in other strategies that translate 
the standards and information from 
assessments into classroom practice for all 
students, including high-need students. 


(continued) 
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Table B-l. Standards and assessments indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 

State was a member 
of a federally-funded 
consortium 
developing 
assessments aligned 
to the Common Core 
State Standards 

Adopted the CCSS in mathematics and 
English language arts and reported 
working with a Federally-funded 
assessment consortium. 

Source: 2011 SEA survey, items 1 and 6 

RTT selection criterion (B)(1) 

The extent to which the State has 
demonstrated its commitment to adopting a 
common set of high-quality standards, 
evidenced by— 

(i) The State's participation in a consortium of 
States that— 

(a) Is working toward jointly developing and 
adopting a common set of K-12 standards that 
are supported by evidence that they are 
internationally benchmarked and build toward 
college and career readiness by the time of 
high school graduation 



RTT selection criterion (B)(2) 

The extent to which the State has 
demonstrated its commitment to improving 
the quality of its assessments, evidenced by the 
State's participation in a consortium of States 
that— 

(i) Is working toward jointly developing and 
implementing common, high-quality 
assessments aligned with the consortium's 
common set of K-12 standards 

District level 

District aware of 
state adoption of 
Common Core 
Standards 

District is in a state that adopted the 
Common Core and district reported that 
its state had adopted the Common Core 
State Standards. 

This indicator is not directly linked to the 
language of the Recovery Act, although it is 
clearly implied (see, for example, ARRA Section 
14003). 


Sources: 2011 SEA survey, item 1 and 
2011 District Survey, item 13 



(continued) 
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Table B-l. Standards and assessments indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 


Components 1 


Recovery Act program requirements 


District distributed 
instructional 
materials or 
provided selection 
guidance on 
curricula aligned 
with the new or 
revised state content 
standards 


District is in a state that adopted the 
Common Core and district distributed or 
provided to schools with at least one of 
the following: 

• Instructional materials aligned with 
new or revised state standards in 
mathematics 

• Instructional materials aligned with 
new or revised state standards in 
reading/English language arts 

• Criteria for schools to use when 
selecting a new curriculum aligned 
with the new or revised state 
standards for mathematics 

• Criteria for schools to use when 
selecting a new curriculum aligned 
with the new or revised state 
standards for reading/English 
language arts. 


Sources: 2011 SEA survey, item 1 and 
2011 District Survey, item 16 


RTT selection criterion (B)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan for supporting a statewide transition to 
and implementation of internationally 
benchmarked K-12 standards that build toward 
college and career readiness by the time of 
high school graduation, and high-quality 
assessments tied to these standards. State or 
LEA activities might, for example, include: ... 
developing or acquiring, disseminating and 
implementing high-quality instructional 
materials and assessments. ..developing or 
acquiring and delivering high-quality 
professional development to support the 
transition to new standards and assessments. 

Investing in Innovation Fund: Notice Inviting 
Applications for New Awards for Fiscal Year (FY) 
2010. Absolute Priority 3— Innovations That 
Complement the Implementation of High 
Standards and High-Quality Assessments 
Under this priority, the Department provides 
funding for practices, strategies, or programs 
that are designed to support States' efforts to 
transition to standards and assessments that 
measure students' progress toward college- 
and career-readiness, including curricular and 
instructional practices, strategies, or programs 
in core academic subjects (as defined in section 
9101(11) of the ESEA) that are aligned with 
high academic content and achievement 
standards and with high quality assessments 
based on those standards. Proposed projects 
may include, but are not limited to, practices, 
strategies, or programs that are designed to: 

(c) translate the standards and information 
from assessments into classroom practices that 
meet the needs of all students, including high- 
need students. 


(continued) 
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Table B-l. Standards and assessments indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 


Components 1 


Recovery Act program requirements 


District provided 
professional 
development on new 
or revised state 
content standards 
for educators who 
teach or mentor 
mathematics or 
reading/English 
language arts 


District is in a state that adopted the 
Common Core and reported providing 
at least one of the following 
professional development activities: 

• In-person professional 
development on the new or revised 
state standards for teachers who 
teach mathematics 

• Internet-based professional 
development on the new or revised 
state standards for educators who 
teach or mentor in mathematics 

• In-person professional 
development on the new or revised 
state standards for teachers who 
teach reading/English language arts 

• Internet-based professional on the 
new or revised state standards for 
educators who teach or mentor in 
reading/English language arts. 


Sources: 2011 SEA survey, item 1 and 
2011 District Survey, item 17 


RTT selection criterion (B)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan for supporting a statewide transition to 
and implementation of internationally 
benchmarked K-12 standards that build toward 
college and career readiness by the time of 
high school graduation, and high-quality 
assessments tied to these standards. State or 
LEA activities might, for example, 
include:. ..developing or acquiring and 
delivering high-quality professional 
development to support the transition to new 
standards and assessments. 

Investing in Innovation Fund: Notice Inviting 
Applications for New Awards for Fiscal Year (FY) 
2010. Absolute Priority 3— Innovations That 
Complement the Implementation of High 
Standards and High-Quality Assessments 
Under this priority, the Department provides 
funding for practices, strategies, or programs 
that are designed to support States' efforts to 
transition to standards and assessments that 
measure students' progress toward college- 
and career-readiness, including curricular and 
instructional practices, strategies, or programs 
in core academic subjects (as defined in section 
9101(11) of the ESEA) that are aligned with 
high academic content and achievement 
standards and with high quality assessments 
based on those standards. Proposed projects 
may include, but are not limited to, practices, 
strategies, or programs that are designed to: 

(c) translate the standards and information 
from assessments into classroom practices that 
meet the needs of all students, including high- 
need students. 


(continued) 
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Table B-l. Standards and assessments indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 


Components 1 


Recovery Act program requirements 


District provided 
professional 
development on new 
or revised state 
content standards 
on instructional 
strategies for 
teachers to help 
English learners or 
students with 
disabilities master 
the content 
standards 


District is in a state that adopted the 
Common Core and reported providing 
at least one of the following 
professional development activities: 

• Internet-based professional 
development on instructional 
strategies for teachers specifically 
designed to help English language 
learners master new or revised 
state content standards 

• In-person professional 
development on instructional 
strategies for teachers specifically 
designed to help English language 
learners master new or revised 
state content standards 

• Internet-based professional 
development on instructional 
strategies for teachers specifically 
designed to help students with 
disabilities master new or revised 
state content standards 

• In-person professional 
development on instructional 
strategies for teachers specifically 
designed to help students with 
disabilities master new or revised 
state content standards. 


Sources: 2011 SEA survey, item 1 and 
2011 District Survey, item 17 


RTT selection criterion (B)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan for supporting a statewide transition to 
and implementation of internationally 
benchmarked K-12 standards that build toward 
college and career readiness by the time of 
high school graduation, and high-quality 
assessments tied to these standards. State or 
LEA activities might, for example, 
include:. ..developing or acquiring and 
delivering high-quality professional 
development to support the transition to new 
standards and assessments. 

Investing in Innovation Fund: Notice Inviting 
Applications for New Awards for Fiscal Year (FY) 
2010. Absolute Priority 3— Innovations That 
Complement the Implementation of High 
Standards and High-Quality Assessments 
Under this priority, the Department provides 
funding for practices, strategies, or programs 
that are designed to support States' efforts to 
transition to standards and assessments that 
measure students' progress toward college- 
and career-readiness, including curricular and 
instructional practices, strategies, or programs 
in core academic subjects (as defined in section 
9101(11) of the ESEA) that are aligned with 
high academic content and achievement 
standards and with high quality assessments 
based on those standards. Proposed projects 
may include, but are not limited to, practices, 
strategies, or programs that are designed to: 

(c) translate the standards and information 
from assessments into classroom practices that 
meet the needs of all students, including high- 
need students. 


(continued) 


B-6 


Table B-l. Standards and assessments indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



Investing in Innovation Fund: Notice Inviting 

Applications for New Awards for Fiscal Year (FY) 
2010. Competitive Preference Prioritv 7— 
Innovations To Address the Unique Learning 
Needs of Students with Disabilities and Limited 
English Proficient Students 
We give competitive preference to applications 
for projects that would implement innovative 
practices, strategies, or programs that are 
designed to address the unique learning needs 
of students with disabilities, including those 
who are assessed based on alternate academic 
achievement standards, or the linguistic and 
academic needs of limited English proficient 
students. To meet this priority, applications 
must provide for the implementation of 
particular practices, strategies, or programs 
that are designed to improve academic 
outcomes, close achievement gaps, and 
increase college- and career-readiness, 
including increasing high school graduation 
rates (as defined in this notice), for students 
with disabilities or limited English proficient 
students. 

Final requirements for School Improvement 
Grants (2010 Notice) (l)(vii) under SIG 
transformation model permissible activities 
"Provide additional supports and professional 
development to teachers and principals in 
order to implement effective strategies to 
support students with disabilities in the least 
restrictive environment and to ensure that 
limited English proficient students acquire 
language skills to master academic content." 


(continued) 
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Table B-l. Standards and assessments indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 


School level 



Teachers received 
professional 
development on new 
and revised state 
content standards 


Reported using at least one of the 

following professional development 

activities: 

• In-person professional development 
for teachers on the new or revised 
content standards for mathematics 

• Internet-based professional 
development for educators on the 
new or revised content standards 
for mathematics 

• In-person professional 
development for teachers on the 
new or revised content standards 
for reading/English language arts 

• Internet-based professional 
development for educators on the 
new or revised content standards 
for reading/English language arts. 

Sources: 2011 School Survey, item 15 


RTT selection criterion (B)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan for supporting a statewide transition to 
and implementation of internationally 
benchmarked K-12 standards that build toward 
college and career readiness by the time of 
high school graduation, and high-quality 
assessments tied to these standards. State or 
LEA activities might, for example, 
include:. ..developing or acquiring and 
delivering high-quality professional 
development to support the transition to new 
standards and assessments. 

Investing in Innovation Fund: Notice Inviting 
Applications for New Awards for Fiscal Year (FY) 
2010. Absolute Priority 3— Innovations That 
Complement the Implementation of High 
Standards and High-Quality Assessments 
Under this priority, the Department provides 
funding for practices, strategies, or programs 
that are designed to support States' efforts to 
transition to standards and assessments that 
measure students' progress toward college- 
and career-readiness, including curricular and 
instructional practices, strategies, or programs 
in core academic subjects (as defined in section 
9101(11) of the ESEA) that are aligned with 
high academic content and achievement 
standards and with high quality assessments 
based on those standards. Proposed projects 
may include, but are not limited to, practices, 
strategies, or programs that are designed to: 

(c) translate the standards and information 
from assessments into classroom practices that 
meet the needs of all students, including high- 
need students. 


(continued) 
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Table B-l. Standards and assessments indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 


Components 1 


Recovery Act program requirements 


Teacher received 
professional 
development 
targeted to help 
English learners or 
students with 
disabilities master 
new or revised state 
content standards 


Reported targeting professional 
development specifically designed to 
help at least one of the following 
student groups: 

• Help English language learners 
master new or revised state 
content standards 

• Help students with disabilities 
master new or revised state 
content standards 


Sources: 2011 School Survey, item 16 


RTT selection criterion (B)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan for supporting a statewide transition to 
and implementation of internationally 
benchmarked K-12 standards that build toward 
college and career readiness by the time of 
high school graduation, and high-quality 
assessments tied to these standards. State or 
LEA activities might, for example, 
include:. ..developing or acquiring and 
delivering high-quality professional 
development to support the transition to new 
standards and assessments. 


Investing in Innovation Fund: Notice Inviting 
Applications for New Awards for Fiscal Year (FY) 
2010. Absolute Priority 3— Innovations That 
Complement the Implementation of High 
Standards and High-Quality Assessments 
Under this priority, the Department provides 
funding for practices, strategies, or programs 
that are designed to support States' efforts to 
transition to standards and assessments that 
measure students' progress toward college- 
and career-readiness, including curricular and 
instructional practices, strategies, or programs 
in core academic subjects (as defined in section 
9101(11) of the ESEA) that are aligned with 
high academic content and achievement 
standards and with high quality assessments 
based on those standards. Proposed projects 
may include, but are not limited to, practices, 
strategies, or programs that are designed to: 

(c) translate the standards and information 
from assessments into classroom practices that 
meet the needs of all students, including high- 
need students. 


(continued) 
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Table B-l. Standards and assessments indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



Investing in Innovation Fund: Notice Inviting 

Applications for New Awards for Fiscal Year (FY) 
2010. Competitive Preference Priority 7— 
Innovations To Address the Unique Learning 
Needs of Students with Disabilities and Limited 
English Proficient Students 
We give competitive preference to applications 
for projects that would implement innovative 
practices, strategies, or programs that are 
designed to address the unique learning needs 
of students with disabilities, including those 
who are assessed based on alternate academic 
achievement standards, or the linguistic and 
academic needs of limited English proficient 
students. To meet this priority, applications 
must provide for the implementation of 
particular practices, strategies, or programs 
that are designed to improve academic 
outcomes, close achievement gaps, and 
increase college- and career-readiness, 
including increasing high school graduation 
rates (as defined in this notice), for students 
with disabilities or limited English proficient 
students. 

Final requirements for School Improvement 
Grants (2010 Notice) (l)(vii) under SIG 
transformation model permissible activities 
"Provide additional supports and professional 
development to teachers and principals in 
order to implement effective strategies to 
support students with disabilities in the least 
restrictive environment and to ensure that 
limited English proficient students acquire 
language skills to master academic content." 


(continued) 
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Table B-l. Standards and assessments indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 


Components 1 


Recovery Act program requirements 


School used 
curriculum materials 
aligned with new or 
revised state content 
standards 


Reported using at least one of the 
following practices to implement new or 
revised state content standards: 

• Curriculum frameworks or pacing 
guides aligned with the new or 
revised standards for mathematics 

• Curriculum aligned with the new or 
revised standards for mathematics 

• Curriculum frameworks or pacing 
guides aligned with the new or 
revised standards for 
reading/English language arts 

• Curriculum aligned with the new or 
revised standards for 
reading/English language arts. 


Sources: 2011 School Survey, item 15 


RTT selection criterion (B)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan for supporting a statewide transition to 
and implementation of internationally 
benchmarked K-12 standards that build toward 
college and career readiness by the time of 
high school graduation, and high-quality 
assessments tied to these standards. State or 
LEA activities might, for example, include: ... 
developing or acquiring, disseminating and 
implementing high-quality instructional 
materials and assessments. 

Investing in Innovation Fund: Notice Inviting 
Applications for New Awards for Fiscal Year (FY) 
2010. Absolute Priority 3— Innovations That 
Complement the Implementation of High 
Standards and High-Quality Assessments 
Under this priority, the Department provides 
funding for practices, strategies, or programs 
that are designed to support States' efforts to 
transition to standards and assessments that 
measure students' progress toward college- 
and career-readiness, including curricular and 
instructional practices, strategies, or programs 
in core academic subjects (as defined in section 
9101(11) of the ESEA) that are aligned with 
high academic content and achievement 
standards and with high quality assessments 
based on those standards. Proposed projects 
may include, but are not limited to, practices, 
strategies, or programs that are designed to: 

(c) translate the standards and information 
from assessments into classroom practices that 
meet the needs of all students, including high- 
need students. 


1 For indicators where the source is 2011 SEA survey, the text listed in the components column is the actual survey item 
wording. 
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Table B-2. Data systems indicators, components, and Recovery Act program requirements, by 
education level 


Indicator 

Components 1 

Recovery Act program requirements 

SEA level 

State operated a 
longitudinal data 
system that 
included 12 core 
components 

Had all 12 components of a statewide 

longitudinal data system outlined in the 

America COMPETES Act: 

• Uses unique student identifier 

• Contains student-level enrollment, 
demographic, and program 
participation information 

• Contains exit, drop out, transfer, and 
completion information of P-16 
programs 

• Communicates with higher education 

• Assesses data quality, validity, 
reliability 

• Contains yearly test records 

• Contains information on students not 
tested 

• Uses teacher identification systems 
and can match them with students 

• Contains student-level transcript 
information 

• Contains college readiness test scores 

• Contains information on students' 
transition from high school to 
postsecondary institutions 

• Contains other information to 
determine alignment and 
preparedness for success in 
postsecondary education 

Source: U.S. Department of Education, 

State Fiscal Stabilization Fund Initial 

Annual State Reports (2009-10) and 

Amended Applications (2010-11) 

RTT selection criterion (C)(1) 

The extent to which the State has a statewide 
longitudinal data system that includes all of 
the America COMPETES Act elements. 

SFSF indicator (b)(1) 

Which of the 12 elements described in 
section 6401(e)(2)(D) of the America 
COMPETES Act (20 U.S.C. 9871) are included 
in the State's statewide longitudinal data 
system. 

SLDS request for applications (NCES 09-02 IV) 
Any statewide, longitudinal data system to be 
supported with funds made available 
pursuant to this competition (Grants for 
Statewide, Longitudinal Data Systems under 
ARRA) must meet the requirements described 
below... Required data system elements: A 
data system developed with funding obtained 
pursuant to this grant competition must 
include at least these 12 elements prescribed 
by the America COMPETES Act: 


(continued) 
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Table B-2. Data systems indicators, components, and Recovery Act program requirements, by 
education level (cont'd) 


Indicator 


Components 1 


Recovery Act program requirements 


State data system 
had ability to link 
teachers to student 
data 


Had a longitudinal data system with 
teacher identifiers and the ability to match 
teachers to students 


RTT selection criterion (C)(1) 

The extent to which the State has a statewide 
longitudinal data system that includes all of 
the America COMPETES Act elements. 


Source: U.S. Department of Education, 
State Fiscal Stabilization Fund Initial 
Annual State Reports (2009-10) and 
Amended Applications (2010-11) 


SFSF indicator (b)(1) 

Which of the 12 elements described in 
section 6401(e)(2)(D) of the America 
COMPETES Act (20 U.S.C. 9871) are included 
in the State's statewide longitudinal data 
system. 


Teacher Incentive Fund core element D (2010 
Application) 2 

A data-management system that can link 
student achievement (as defined in this 
notice) data to teacher and principal payroll 
and human resources systems. 


SLDS request for applications (NCES 09-02 IV) 
Any statewide, longitudinal data system to be 
supported with funds made available 
pursuant to this competition (Grants for 
Statewide, Longitudinal Data Systems under 
ARRA) must meet the requirements described 
below... Required data system capabilities: 

The system must link student data with 
teachers, i.e., it must enable the matching of 
teachers and students so that a given student 
may be matched with the particular teachers 
primarily responsible for providing instruction 
in various subjects. 


(continued) 
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Table B-2. Data systems indicators, components, and Recovery Act program requirements, by 
education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 

SEA facilitated 
educators' access to 
assessment data 

Reported at least one of the following to 
facilitate access to new or existing 
assessment data: 

• Providing educators with key LEA, 
school and student indicators through 
report cards, data dashboards, or other 
feedback and analysis systems 

• Establishing and maintaining state data 
systems that share longitudinal data on 
students with local data systems 

Source: 2011 SEA survey, item 9 

RTT selection criterion (C)(2) 

The extent to which the State has a high- 
quality plan to ensure that data from the 
State's statewide longitudinal data system are 
accessible to, and used to inform and engage, 
as appropriate, key stakeholders (e.g., 
parents, students, teachers, principals, LEA 
leaders, community members, unions, 
researchers, and policymakers); and that the 
data support decision-makers in the 
continuous improvement of efforts in such 
areas as policy, instruction, operations, 
management, resource allocation, and overall 
effectiveness. 

RTT selection criterion (C)(3) 

The extent to which the State, in 
collaboration with its participating LEAs, has a 
high-quality plan to— 

(i) Increase the acquisition, adoption, and use 
of local instructional improvement systems 
that provide teachers, principals, and 
administrators with the information and 
resources they need to inform and improve 
their instructional practices, decision-making, 
and overall effectiveness; 

(iii) Make the data from instructional 
improvement systems, together with 
statewide longitudinal data system data, 
available and accessible to researchers so 
that they have detailed information with 
which to evaluate the effectiveness of 
instructional materials, strategies, and 
approaches for educating different types of 
students (e.g., students with disabilities, 
English language learners, students whose 
achievement is well below or above grade 
level). 

SLDS request for applications (NCES 09-02 IV) 
Any statewide, longitudinal data system to be 
supported with funds made available 
pursuant to this competition (Grants for 
Statewide, Longitudinal Data Systems under 


(continued) 
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Table B-2. Data systems indicators, components, and Recovery Act program requirements, by 
education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



ARRA) must meet the requirements described 
below... Required data system capabilities: 

The system must enable data to be easily 
generated for continuous improvement and 
decision-making, including timely reporting to 
parents, teachers, and school leaders on the 
achievement of their students. 

SEA provided 
professional 
development or 
technical assistance 
to support 
educators' use of 
assessment data 

Reported providing at least one of the 
following through state-developed 
professional development or 
guidance/technical assistance to LEAs on: 

• Strategies and procedures for LEA staff 
to use in accessing new or existing state 
assessment data 

• Use of new or existing assessment data 
by teachers to improve instruction 
(including interim/formative 
assessments) 

• Use of new or existing assessment data 
by principals and school leaders in 
school improvement planning 

RTT selection criterion (C)(3) 

The extent to which the State, in 
collaboration with its participating LEAs, has a 
high-quality plan to— ... 

(ii) Support participating LEAs and schools 
that are using instructional improvement 
systems in providing effective professional 
development to teachers, principals and 
administrators on how to use these systems 
and the resulting data to support continuous 
instructional improvement ... 


Source: 2011 SEA survey, item 9 


District level 

District used 
longitudinal data to 
track student 
achievement gains 
for individual 
teachers 

Reported the following strategy for using 
assessment results: 

• Using longitudinal data to track 
student achievement gains for 
individual teachers. 

Source: 2011 District Survey, item 21 

RTT selection criterion (C)(2) 

The extent to which the State has a high- 
quality plan to ensure that data from the 
State's statewide longitudinal data system are 
accessible to, and used to inform and engage, 
as appropriate, key stakeholders (e.g., 
parents, students, teachers, principals, LEA 
leaders, community members, unions, 
researchers, and policymakers); and that the 
data support decision-makers in the 
continuous improvement of efforts in such 
areas as policy, instruction, operations, 
management, resource allocation, and overall 
effectiveness. 


(continued) 
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Table B-2. Data systems indicators, components, and Recovery Act program requirements, by 
education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



Investing in Innovation Fund: Notice Inviting 



Applications for New Awards for Fiscal Year 
(FY) 2010. Absolute Priority 2— Innovations 



That Improve the Use of Data 
Under this priority, the Department provides 
funding to support strategies, practices, or 
programs that are designed to (a) encourage 
and facilitate the evaluation, analysis, and use 
of student achievement or student growth 
data by educators, families, and other 
stakeholders in order to inform decision 
making and improve student achievement, 
student growth, or teacher, principal, school, 
or LEA performance and productivity. 

District provided 
educators' with 
access to 
assessment data 

Reported using at least one of the 

following strategies for using assessment 

results: 

• Providing teachers with on-line access 
to individual student results from 
state summative assessments 

• Providing teachers with on-line access 
to individual student results from 
district summative assessments 

• Providing teachers with on-line access 
to individual student results from 
interim assessments 

• Providing educators with key 
aggregate student and school 
indicators through report cards, data 
dashboards, or other feedback and 
analysis systems. 

Sources: 2011 District Survey, item 21 

RTT selection criterion (C)(3) 

The extent to which the State, in 
collaboration with its participating LEAs, has a 
high-quality plan to— 

(i) Increase the acquisition, adoption, and use 
of local instructional improvement systems 
that provide teachers, principals, and 
administrators with the information and 
resources they need to inform and improve 
their instructional practices, decision-making, 
and overall effectiveness; 

(iii) Make the data from instructional 
improvement systems, together with 
statewide longitudinal data system data, 
available and accessible to researchers so 
that they have detailed information with 
which to evaluate the effectiveness of 
instructional materials, strategies, and 
approaches for educating different types of 
students (e.g., students with disabilities, 
English language learners, students whose 
achievement is well below or above grade 
level). 


(continued) 
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Table B-2. Data systems indicators, components, and Recovery Act program requirements, by 
education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



Final requirements for School Improvement 
Grants (2010 Notice) (l)(vii) under SIG 
turnaround model and (2)(i)(B) under SIG 
transformation model permissible activities 
"Promote the continuous use of student data 
(such as from formative, interim, and 
summative assessments) to inform and 
differentiate instruction in order to meet the 
academic needs of individual students." 

District provided 
educators' with 
professional 
development on the 
use of assessment 
data for 
instructional 
planning 

Reported providing at least one of the 

following 

professional development activities: 

• Providing professional development 
focused on using summative 
assessment results for instructional 
planning 

• Providing professional development 
focused on using interim assessment 
results for instructional planning. 

Source: 2011 District Survey, item 11 

Promoted by the Recovery Act via: 

RTT selection criterion (C)(3) 

The extent to which the State, in 
collaboration with its participating LEAs, has a 
high-quality plan to— 

(i) Increase the acquisition, adoption, and use 
of local instructional improvement systems 
that provide teachers, principals, and 
administrators with the information and 
resources they need to inform and improve 
their instructional practices, decision-making, 
and overall effectiveness; 

(iii) Make the data from instructional 
improvement systems, together with 
statewide longitudinal data system data, 
available and accessible to researchers so 
that they have detailed information with 
which to evaluate the effectiveness of 
instructional materials, strategies, and 
approaches for educating different types of 
students (e.g., students with disabilities, 
English language learners, students whose 
achievement is well below or above grade 
level). 


(continued) 
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Table B-2. Data systems indicators, components, and Recovery Act program requirements, by 
education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 

School level 

School used 

Reported the following assessment data 

RTT selection criterion (C)(2) 

longitudinal data to 

practice: 

The extent to which the State has a high- 

track student 

• Using longitudinal data to track 

quality plan to ensure that data from the 

achievement gains 

student achievement gains for 

State's statewide longitudinal data system are 

for individual 

individual teachers. 

accessible to, and used to inform and engage, 

teachers 

Source: 2011 School Survey, item 18 

as appropriate, key stakeholders (e.g., 
parents, students, teachers, principals, LEA 
leaders, community members, unions, 
researchers, and policymakers); and that the 
data support decision-makers in the 
continuous improvement of efforts in such 
areas as policy, instruction, operations, 
management, resource allocation, and overall 
effectiveness. 



Investing in Innovation Fund: Notice Inviting 



Applications for New Awards for Fiscal Year 
(FY) 2010. Absolute Priority 2— Innovations 



That Improve the Use of Data 
Under this priority, the Department provides 
funding to support strategies, practices, or 
programs that are designed to (a) encourage 
and facilitate the evaluation, analysis, and use 
of student achievement or student growth 
data by educators, families, and other 
stakeholders in order to inform decision 
making and improve student achievement, 
student growth, or teacher, principal, school, 
or LEA performance and productivity. 

Teachers had on- 

Reported that teachers have on-line access 

RTT selection criterion (C)(3) 

line access to 

to individual student results from at least 

The extent to which the State, in 

student assessment 

one of the following: 

collaboration with its participating LEAs, has a 

results 

• state summative assessments 

high-quality plan to— 


• district summative assessments 

(i) Increase the acquisition, adoption, and use 


• interim assessments. 

of local instructional improvement systems 
that provide teachers, principals, and 


Source: 2011 School Survey, item 18 

administrators with the information and 
resources they need to inform and improve 
their instructional practices, decision-making, 
and overall effectiveness; 


(continued) 
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Table B-2. Data systems indicators, components, and Recovery Act program requirements, by 
education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



(iii) Make the data from instructional 
improvement systems, together with 
statewide longitudinal data system data, 
available and accessible to researchers so 
that they have detailed information with 
which to evaluate the effectiveness of 
instructional materials, strategies, and 
approaches for educating different types of 
students (e.g., students with disabilities, 
English language learners, students whose 
achievement is well below or above grade 
level). 

Final requirements for School Improvement 
Grants (2010 Notice) (l)(vii) under SIG 
turnaround model and (2)(i)(B) under SIG 
transformation model permissible activities 
"Promote the continuous use of student data 
(such as from formative, interim, and 
summative assessments) to inform and 
differentiate instruction in order to meet the 
academic needs of individual students." 

School used student 
assessment data to 
tailor instruction 

Reported the following strategy for 
improving instruction: 

• Using student assessment data to 
tailor instruction in the classroom. 

Source: 2011 School Survey, item 11 

RTT selection criterion (C)(3) 

The extent to which the State, in 
collaboration with its participating LEAs, has a 
high-quality plan to— 

(i) Increase the acquisition, adoption, and use 
of local instructional improvement systems 
that provide teachers, principals, and 
administrators with the information and 
resources they need to inform and improve 
their instructional practices, decision-making, 
and overall effectiveness; 

(iii) Make the data from instructional 
improvement systems, together with 
statewide longitudinal data system data, 
available and accessible to researchers so 
that they have detailed information with 
which to evaluate the effectiveness of 
instructional materials, strategies, and 
approaches for educating different types of 
students (e.g., students with disabilities. 
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Table B-2. Data systems indicators, components, and Recovery Act program requirements, by 
education level s (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



English language learners, students whose 
achievement is well below or above grade 
level). 

Final requirements for School Improvement 
Grants (2010 Notice) (l)(vii) under SIG 
turnaround model and (2)(i)(B) under SIG 
transformation model permissible activities 
"Promote the continuous use of student data 
(such as from formative, interim, and 
summative assessments) to inform and 
differentiate instruction in order to meet the 
academic needs of individual students." 

School used student 
assessment data to 
identify students for 
additional support 

Reported the following strategy for 
improving instruction and related student 
services: 

• Using student assessment data to 
identify students for additional 
support. 

Source: 2011 School Survey, item 11 

RTT selection criterion (C)(3) 

The extent to which the State, in 
collaboration with its participating LEAs, has a 
high-quality plan to— 

(i) Increase the acquisition, adoption, and use 
of local instructional improvement systems 
that provide teachers, principals, and 
administrators with the information and 
resources they need to inform and improve 
their instructional practices, decision-making, 
and overall effectiveness; 

(iii) Make the data from instructional 
improvement systems, together with 
statewide longitudinal data system data, 
available and accessible to researchers so 
that they have detailed information with 
which to evaluate the effectiveness of 
instructional materials, strategies, and 
approaches for educating different types of 
students (e.g., students with disabilities, 
English language learners, students whose 
achievement is well below or above grade 
level). 


(continued) 
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Table B-2. Data systems indicators, components, and Recovery Act program requirements, by 
education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



Final requirements for School Improvement 
Grants (2010 Notice) (lHvii) under SIG 
turnaround model and (2)(i)(B) under SIG 
transformation model permissible activities 
"Promote the continuous use of student data 
(such as from formative, interim, and 
summative assessments) to inform and 
differentiate instruction in order to meet the 
academic needs of individual students." 


1 For indicators where the source is 2011 SEA survey, the text listed in the components column is the actual survey item 
wording. 

2 Given the Teacher Incentive Fund program's emphasis on this component of longitudinal data systems, it was examined as a 
separate indicator in addition to being included in the previous indicator. 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level 


Indicator 

Components 1 

Recovery Act program requirements 


SEA level 


SEA 

Reported at least one of the following: 

RTT selection criterion (D)(1) 

simplified/shortened 

• Simplifying or shortening the process of 

The extent to which the State has— 

educator licensure 

obtaining full licensure and/or 

(i) Legal, statutory, or regulatory 

process or authorized 

certification (e.g., require fewer credit 

provisions that allow alternative routes* 

non-university 

hours)for: 

to certification for teachers and principals, 

preparation programs 

o State university based teacher 

particularly routes that allow for providers 


preparation programs 

in addition to institutions of higher 


o Alternative pathway teacher 

education 


preparation programs 

(ii) Alternative routes to certification that 


• Authorizing independent providers 

are in use 


Source: 2011 SEA survey, item 23 

* definition includes "significantly limit the 
amount of coursework required or have 
options to test out of courses." 

SEA issued standards or 

Reported issuing standards or guidelines 

RTT selection criterion (B)(3) 

guidelines for teacher 

for pre-service teacher preparation 

The extent to which the State, in 

preparation programs 

programs in at least one of the following: 

collaboration with its participating LEAs, 


• Alignment with new or revised state 

has a high-quality plan for supporting a 


content standards 

statewide transition to and 


• Alignment with state teacher standards 

implementation of internationally 


• Provision of training on practices 

benchmarked K-12 standards ... State or 


specifically related to improving low- 

LEA activities might, for example, include: 


performing schools 

...developing or acquiring and delivering 


• Tracking the effectiveness of graduates 

high-quality professional development to 


based on student achievement gains 

support the transition to new standards 


and making these data publicly 
available 

and assessments. 

RTT selection criterion (D)(4) 


Source: 2011 SEA survey, item 23 

The extent to which the State has a high- 
quality plan and ambitious yet achievable 
annual targets to— 

(i) Link student achievement and student 
growth (both as defined in this notice) 
data to the students' teachers and 
principals, to link this information to the 
in-State programs where those teachers 
and principals were prepared for 
credentialing, and to publicly report the 
data for each credentialing program in the 
State; 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



Because states have a traditional role in 
reviewing and approving educator 
preparation programs, it was relevant for 
the study to ask states what they were 
doing to align preparation programs with 
these standards. 

SEA issued standards or 
guidelines for principal 
preparation programs 

Reported issuing standards or guidelines 
for pre-service principal/school leader 
preparation programs to promote at least 
one of the following: 

• Alignment with new or revised state 
content standards 

• Alignment with state principal/school 
leader standards 

• Provision of training on practices 
specifically related to improving low- 
performing schools 

• Tracking the effectiveness of graduates 
based on student achievement gains 
and making the these data publicly 
available 

Source: 2011 SEA survey, item 23 

RTT selection criterion (B)(3) 

The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan for supporting a 
statewide transition to and 
implementation of internationally 
benchmarked K-12 standards ... State or 
LEA activities might, for example, include: 
...developing or acquiring and delivering 
high-quality professional development to 
support the transition to new standards 
and assessments. 

RTT selection criterion (D)(4) 

The extent to which the State has a high- 
quality plan and ambitious yet achievable 
annual targets to— 

(i) Link student achievement and student 
growth (both as defined in this notice) 
data to the students' teachers and 
principals, to link this information to the 
in-State programs where those teachers 
and principals were prepared for 
credentialing, and to publicly report the 
data for each credentialing program in the 
State. 

SEA supported use of 
multi-level ratings, 
multiple observations, 
and student 
achievement gains for 
teacher evaluation 

Reported at least one of the following 
role(s): 

• Administering a state-developed 
teacher evaluation system in which LEA 
participation is required 

• Administering a state-developed 
teacher evaluation system in which LEA 
participation is optional 

RTT eligibility requirement (b) 

At the time the State submits its 
application, there are no legal, statutory, 
or regulatory barriers at the State level to 
linking data on student achievement (as 
defined in this notice) or student growth 
(as defined in this notice) to teachers and 
principals for the purpose of teacher and 
principal evaluation. 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 


• Setting evaluation system standards 
and guidelines for LEA-designed teacher 
evaluation systems that are required 

• Setting evaluation system standards 
and guidelines for LEA-designed teacher 
evaluation systems that are optional 

• Providing guidance and technical 
assistance to LEAs on the design and 
implementation of their systems 

• Requiring LEAs to submit teacher 
evaluation design and implementation 
plans for SEA approval 

• Requiring LEAs to report on teacher 
evaluation system operations and 
effectiveness 

And included all of the following 

component(s) in the state evaluation 

system, standards, guidance, or technical 

assistance: 

• A rating scale/rubric with three or more 
performance levels to evaluate 
instruction/practice 

• At least two yearly observations of 
classroom instruction with written 
feedback 

• Student achievement gains in NCLB 
grades/subjects used in determining 
individual teacher performance ratings 

Source: 2011 SEA survey, items 15 and 16 

RTT selection criterion (D)(2) 

The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan and ambitious yet 
achievable annual targets to ensure that 
participating LEAs— 

(ii) Design and implement rigorous, 
transparent, and fair evaluation systems 
for teachers and principals that (a) 
differentiate effectiveness using multiple 
rating categories that take into account 
data on student growth as a significant 
factor, and (b) are designed and 
developed with teacher and principal 
involvement; 

SFSF indicator (a)(3) 

Whether the systems used to evaluate the 
performance of teachers include student 
achievement outcomes or student growth 
data as an evaluation criterion. 

Teacher Incentive Fund core element C 
(2010 Application) 

Rigorous, transparent, and fair evaluation 
systems for teachers and principals that 
differentiate effectiveness using multiple 
rating categories that take into account 
student growth (as defined in this notice) 
as a significant factor, as well as classroom 
observations conducted at least twice 
during the school year. 

SEA supported use of 
student achievement 
gains for principal 
evaluation 

Reported at least one of the following 
roles: 

• Administering a state principal 
evaluation system in which LEA 
participation is required 

• Administering a state principal 
evaluation system in which LEA 
participation is optional 

RTT eligibility requirement (b) 

At the time the State submits its 
application, there are no legal, statutory, 
or regulatory barriers at the State level to 
linking data on student achievement (as 
defined in this notice) or student growth 
(as defined in this notice) to teachers and 
principals for the purpose of teacher and 
principal evaluation. 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 


• Setting principal evaluation system 
standards and guidelines that LEA- 
designed systems are required to meet 

• Setting principal evaluation system 
standards and guidelines that LEA- 
designed systems may choose to meet 

• Providing guidance and technical 
assistance to LEAs on system design and 
implementation, including providing 
model principal evaluation systems that 
LEAs may choose to use all or in part 

• Requiring LEAs to submit principal 
evaluation system design and 
implementation plans for SEA approval 

• Requiring LEAs to report on principal 
evaluation system operations and 
effectiveness 

And included the following component in 

the state evaluation system, standards, 

guidance, or technical assistance: 

• Student achievement gains or growth 
used to determine principals' 
performance ratings 

Source: 2011 SEA survey , items 17 and 18 

RTT selection criterion (D)(2) 

The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan and ambitious yet 
achievable annual targets to ensure that 
participating LEAs— 

(ii) Design and implement rigorous, 
transparent, and fair evaluation systems 
for teachers and principals that (a) 
differentiate effectiveness using multiple 
rating categories that take into account 
data on student growth as a significant 
factor, and (b) are designed and 
developed with teacher and principal 
involvement; 

SFSF indicator (a)(6) 

Whether the systems used to evaluate the 
performance of principals include student 
achievement outcomes or student growth 
data as an evaluation criterion. 

Teacher Incentive Fund core element C 
(2010 Application) 

Rigorous, transparent, and fair evaluation 
systems for teachers and principals that 
differentiate effectiveness using multiple 
rating categories that take into account 
student growth (as defined in this notice) 
as a significant factor, as well as classroom 
observations conducted at least twice 
during the school year. 

SEA supported 
differentiating teacher 
compensation based 
on student 
achievement gains 

Reported at least one of the following 

roles: 

• Administering a state-developed 
teacher compensation system in which 
LEA participation is required or optional 

• Setting teacher compensation system 
standards and guidelines that are either 
required or optional for LEAs 

• Providing guidance and technical 

RTT selection criterion (D)(2) 

The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan and ambitious yet 
achievable annual targets to ensure that 
participating LEAs— 

(iv) Use these evaluations, at a minimum, 
to inform decisions regarding— 

(b) Compensating, promoting, and 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 


assistance to LEAs on teacher 
compensation system design and 
implementation 

• Requiring LEAs to submit teacher 
compensation system design and 
implementation plans for SEA approval 

• Requiring LEAs to report on teacher 
compensation system operations and 
effectiveness 

And included one or more of the following 

components in state-developed system, 

standards, guidelines, or technical 

assistance: 

• Base pay increases, add-ons, or stipends 
based in part on achievement gains of 
students in individual teachers 1 classes 

• One-time bonuses for achievement 
gains of students in individual teachers 1 
classes 

• One-time bonuses for achievement 
gains of students served by teacher 
teams 

• One-time bonuses for average 
achievement gains of all students in a 
school 

Source: 2011 SEA survey, items 19 and 20 

retaining teachers and principals, 
including by providing opportunities for 
highly effective teachers and principals 
(both as defined in this notice) to obtain 
additional compensation and be given 
additional responsibilities; 

Teacher Incentive Fund Absolute Priority 1 
(2010 Application) 

Priority 1 (Absolute) -- Differentiated 
Levels of Compensation for Effective 
Teachers and Principals: 

To meet this absolute priority, an 
applicant must demonstrate, in its 
application, that it will develop and 
implement a performance-based 
compensation system (PBCS) that 
rewards, at differentiated levels, teachers 
and principals who demonstrate their 
effectiveness by improving student 
achievement (as defined in this notice) as 
part of the coherent and integrated 
approach of the local educational agency 
(LEA) to strengthening the educator 
workforce. 

District level 

District provided school 
leaders with 
professional 
development or 
flexibility to hire 
effective teachers 

Reported using at least one of the 
following strategies to support teacher 
recruitment and hiring: 

• Focus recruitment efforts on new 
teachers from university-based 
preparation programs that have 
evidence of the effectiveness of 
its graduates based on their 
students' achievement gains 

• Make available or provide and 
hire effective teachers 

Promoted by the Recovery Act via: 

Final requirements for School 
Improvement Grants (2010 Notice) and 
Interventions under RTT Selection 
Criterion (E)(2): Permissible district 
activities under the SIG transformation 
model 

C) Ensuring that the school is not required 
to accept a teacher without the (mutual 
consent of the teacher and principal, 
regardless of the teacher's seniority. 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 


• Make available or provide ongoing 
professional development for principals 
on how they can identify, recruit, and 
hire effective teachers 

• Provide school leaders with the 
authority to hire more qualified transfer 
candidates without regard to district 
seniority status 

Sources: 2011 District Survey, items 2 and 

3 

RTT Priority 6: Invitational Priority 

The Secretary is particularly interested in 
applications in which the State's 
participating LEAs seek to create the 
conditions for reforms and innovation as 
well as the conditions for learning by 
providing schools with flexibility and 
autonomy in such areas as (i) selecting 
staff. 

Investing in Innovation Fund; Notice 
Inviting Applications for New Awards for 
Fiscal Year (FY) 2010, Absolute Priority 1— 
Innovations That Support Effective 
Teachers and Principals 
Under this priority, the Department 
provides funding to support practices, 
strategies, or programs that are designed 
to increase the number or percentages of 
teachers or principals who are highly 
effective teachers or principals or reduce 
the number or percentages of teachers or 
principals who are ineffective, especially 
for teachers of high-need students, by ... 
recruiting ...highly effective teachers or 
principals. 

District operated a 
teacher evaluation 
system that included 
multi-level rubrics, 
multiple observations, 
and student 
achievement gains 

Reported that its educator performance 
evaluation system included all of the 
following components: 

• Teacher evaluation system uses a rating 
scale or rubric that defines three or 
more performance levels to evaluate 
classroom instruction or practice 

• Teacher evaluation system includes at 
least two yearly observations of 
classroom instruction with written 
feedback 

Teacher Incentive Fund core element C 
(2010 Application) 

Rigorous, transparent, and fair evaluation 
systems for teachers and principals that 
differentiate effectiveness using multiple 
rating categories that take into account 
student growth (as defined in this notice) 
as a significant factor, as well as classroom 
observations conducted at least twice 
during the school year. 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 


• Teacher evaluation system includes 
student achievement gains in NCLB 
grades/subjects in determining 
individual teacher performance ratings 

Sources: 2011 District Survey, item 5 

RTT selection criterion (D)(2) 

The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan and ambitious yet 
achievable annual targets to ensure that 
participating LEAs— 

(ii) Design and implement rigorous, 
transparent, and fair evaluation systems 
for teachers and principals that (a) 
differentiate effectiveness using multiple 
rating categories that take into account 
data on student growth as a significant 
factor, and (b) are designed and 
developed with teacher and principal 
involvement; 

SFSF indicator (a)(3) 

Whether the systems used to evaluate the 
performance of teachers include student 
achievement outcomes or student growth 
data as an evaluation criterion. 

(note that SFSF reporting requirements 
provide for district reporting of whether 
the systems used to evaluate the 
performance of teachers include student 
achievement outcomes as an evaluation 
criterion; see State Fiscal Stabilization 
Fund: Summary of Proposed Data and 
Information Requirements. U.S. 
Department of Education 
July 29, 2009) 

Investing in Innovation Fund; Notice 
Inviting Applications for New Awards for 
Fiscal Year (FY) 2010, Absolute Priority 1- 
Innovations That Support Effective 
Teachers and Principals 
Under this priority, the Department 
provides funding to support practices, 
strategies, or programs that are designed 
to increase the number or percentages of 
teachers or principals who are highly 
effective teachers or principals or reduce 
the number or percentages of teachers or 
principals who are ineffective, especially 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



for teachers of high-need students, by 
identifying ... highly effective teachers or 
principals (or removing ineffective 
teachers or principals). In such initiatives, 
teacher or principal effectiveness should 
be determined through an evaluation 
system that is rigorous, transparent, and 
fair; performance should be differentiated 
using multiple rating categories of 
effectiveness; multiple measures of 
effectiveness should be taken into 
account, with data on student growth as a 
significant factor. 

Final requirements for School 
Improvement Grants (2010 Notice) and 
Interventions under RTT Selection 
Criterion (E)(2) Required district activities 
under the SIG transformation model 
(i) Required activities. The LEA must— 

(B) Use rigorous, transparent, and 
equitable evaluation systems for teachers 
and principals that— 

(1) Take into account data on student 
growth as a significant factor as well as 
other factors such as multiple 
observation-based assessments of 
performance and ongoing collections of 
professional practice reflective of student 
achievement and increased high school 
graduations rates. 

District operated a 
principal evaluation 
system that included 
student achievement 
gains 

Reported that its educator performance 
evaluation system included the following 
component: 

• Principal evaluation system includes 
student achievement gains or growth in 
determining principals' performance 
ratings 

Sources: 2011 District Survey, item 5 

Teacher Incentive Fund core element C 
(2010 Application) 

Rigorous, transparent, and fair evaluation 
systems for teachers and principals that 
differentiate effectiveness using multiple 
rating categories that take into account 
student growth (as defined in this notice) 
as a significant factor, as well as classroom 
observations conducted at least twice 
during the school year. 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



RTT selection criterion (D)(2) 

The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan and ambitious yet 
achievable annual targets to ensure that 
participating LEAs— 

(ii) Design and implement rigorous, 
transparent, and fair evaluation systems 
for teachers and principals that (a) 
differentiate effectiveness using multiple 
rating categories that take into account 
data on student growth as a significant 
factor, and (b) are designed and 
developed with teacher and principal 
involvement; 

SFSF indicator (a)(6) 

Whether the systems used to evaluate the 
performance of principals include student 
achievement outcomes or student growth 
data as an evaluation criterion. 

(Note that SFSF reporting requirements 
provide for district reporting of whether 
the systems used to evaluate the 
performance of principals include student 
achievement outcomes as an evaluation 
criterion; see State Fiscal Stabilization 
Fund: Summary of Proposed Data and 
Information Requirements. U.S. 
Department of Education 
July 29, 2009) 

Investing in Innovation Fund; Notice 
Inviting Applications for New Awards for 
Fiscal Year (FY) 2010, Absolute Priority 1-- 
Innovations That Support Effective 
Teachers and Principals 
Under this priority, the Department 
provides funding to support practices, 
strategies, or programs that are designed 
to increase the number or percentages of 
teachers or principals who are highly 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



effective teachers or principals or reduce 
the number or percentages of teachers or 
principals who are ineffective, especially 
for teachers of high-need students, by 
identifying ... highly effective teachers or 
principals (or removing ineffective 
teachers or principals). In such initiatives, 
teacher or principal effectiveness should 
be determined through an evaluation 
system that is rigorous, transparent, and 
fair; performance should be differentiated 
using multiple rating categories of 
effectiveness; multiple measures of 
effectiveness should be taken into 
account, with data on student growth as a 
significant factor. 

Final requirements for School 
Improvement Grants (2010 Notice) and 
Interventions under RTT Selection 
Criterion (E)(2) Required district activities 
under the SIG transformation model 
(i) Required activities. The LEA must— 

(B) Use rigorous, transparent, and 
equitable evaluation systems for teachers 
and principals that— 

(1) Take into account data on student 
growth as a significant factor as well as 
other factors such as multiple 
observation-based assessments of 
performance and ongoing collections of 
professional practice reflective of student 
achievement and increased high school 
graduations rates. 

District differentiated 
teacher compensation 
based on student 
achievement gains 

Reported that its educator compensation 
system included at least one of the 
following components: 

• Teacher compensation system provides 
base pay increases, add-ons, or 
stipends to teachers based in part on 
the achievement gains of students in 
individual teachers' classes 

Teacher Incentive Fund Absolute Priority 1 
(2010 Application) 

Priority 1 (Absolute) -- Differentiated 
Levels of Compensation for Effective 
Teachers and Principals: 

To meet this absolute priority, an 
applicant must demonstrate, in its 
application, that it will develop and 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 


• Teacher compensation system provides 
one-time bonuses for achievement 
gains of students in individual teachers' 
classes 

• Teacher compensation system provides 
one-time bonuses for the achievement 
gains of students served by teacher 
grade-level or other teams 

• Teacher compensation system provides 
one-time bonuses for achievement 
gains of students school-wide 

Source: 2011 District Survey, item 6 

implement a performance-based 
compensation system (PBCS) that 
rewards, at differentiated levels, teachers 
and principals who demonstrate their 
effectiveness by improving student 
achievement (as defined in this notice) as 
part of the coherent and integrated 
approach of the local educational agency 
(LEA) to strengthening the educator 
workforce. 

RTT selection criterion (D)(2) 

The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan and ambitious yet 
achievable annual targets to ensure that 
participating LEAs— 

(ii) Design and implement rigorous, 
transparent, and fair evaluation systems 
for teachers and principals that (a) 
differentiate effectiveness using multiple 
rating categories that take into account 
data on student growth as a significant 
factor, and (b) are designed and 
developed with teacher and principal 
involvement; 

(iv) Use these evaluations, at a minimum, 
to inform decisions regarding— 

(b) Compensating, promoting, and 
retaining teachers and principals, 
including by providing opportunities for 
highly effective teachers and principals 
(both as defined in this notice) to obtain 
additional compensation and be given 
additional responsibilities; 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



Final requirements for School 
Improvement Grants (2010 Notice) and 
Interventions under RTT Selection 
Criterion (E)(2) : Required district activities 

under the SIG transformation model 
(i) Required activities. The LEA must— 

(C) Identify and reward school leaders, 
teachers and other staff who, in 
implementing this model, have increased 
student achievement and high school 
graduation rates and identify and remove 
those who, after ample opportunities 
have been provided for them to improve 
their professional development, have not 
done so; (also a permissible activity under 
the Turnaround model). 

District used student 
achievement gains for 
teacher tenure, 
dismissal, or 
assignment decisions 

Reported that it used at least one of the 
following in educator tenure, assignment 
and retention decisions: 

• Use gains or growth in the achievement 
of teachers' students in deciding 
teacher tenure 

• Use gains or growth in the achievement 
of teachers' students in deciding 
teacher dismissal or non-retention with 
the district 

• Use gains or growth in the achievement 
of teachers' students in deciding 
teacher retention in the school or 
reassignment to another school 

Source: 2011 District Survey, item 7 

RTT Selection Criterion (D)(2)(ii) and (iv)(c) 
and (d))— 

The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan and ambitious yet 
achievable annual targets to ensure that 
participating LEAs— 

(ii) Design and implement rigorous, 
transparent, and fair evaluation systems 
for teachers and principals that (a) 
differentiate effectiveness using multiple 
rating categories that take into account 
data on student growth as a significant 
factor, and (b) are designed and 
developed with teacher and principal 
involvement; 

(iv) use these evaluations, at a minimum, 
to inform decisions regarding (c) whether 
to grant tenure and/or full certification 
(where applicable) to teachers and 
principals using rigorous standards and 
streamlined, transparent, and fair 
procedures; and (d) removing 


(continued) 


B-33 


Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



ineffective tenured and untenured 
teachers and principals after they have 
had ample opportunities to improve, and 
ensuring that such decisions are made 
using rigorous standards and streamlined, 
transparent, and fair procedures. 

SFSF Descriptors (a)(1) 

The systems used to evaluate the 
performance of teachers and the use of 
results from those systems in decisions 
regarding teacher development, 
compensation, promotion, retention, and 
removal. 

Investing in Innovation Fund; Notice 
Inviting Applications for New Awards for 
Fiscal Year (FY) 2010, Absolute Priority 1— 
Innovations That Support Effective 
Teachers and Principals 
Under this priority, the Department 
provides funding to support practices, 
strategies, or programs that are designed 
to increase the number or percentages of 
teachers or principals who are highly 
effective teachers or principals or reduce 
the number or percentages of teachers or 
principals who are ineffective, especially 
for teachers of high-need students, by 
identifying ... highly effective teachers or 
principals (or removing ineffective 
teachers or principals). In such initiatives, 
teacher or principal effectiveness should 
be determined through an evaluation 
system that is rigorous, transparent, and 
fair; performance should be differentiated 
using multiple rating categories of 
effectiveness; multiple measures of 
effectiveness should be taken into 
account, with data on student growth as a 
significant factor. 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 

School level 

Teacher evaluation 
practices included 
multi-level rubrics, 
multiple observations, 
and student 
achievement gains 

Reported that its educator performance 
evaluation practices included all of the 
following: 

• Teacher evaluation practices use a 
rating scale or rubric that defines three 
or more performance levels to evaluate 
classroom instruction or practice 

• Teacher evaluation practices include at 
least two yearly observations of 
classroom instruction with written 
feedback 

• Teacher evaluation practices include 
student achievement gains in NCLB 
grades/subjects in determining 
individual teacher performance ratings 

Source: 2011 School Survey, item 6 

Promoted by the Recovery Act via: 

Teacher Incentive Fund core element C 
(2010 Application) 

Rigorous, transparent, and fair evaluation 
systems for teachers and principals that 
differentiate effectiveness using multiple 
rating categories that take into account 
student growth (as defined in this notice) 
as a significant factor, as well as classroom 
observations conducted at least twice 
during the school year. 

RTT selection criterion (D)(2) 

The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan and ambitious yet 
achievable annual targets to ensure that 
participating LEAs— 

(ii) Design and implement rigorous, 
transparent, and fair evaluation systems 
for teachers and principals that (a) 
differentiate effectiveness using multiple 
rating categories that take into account 
data on student growth as a significant 
factor, and (b) are designed and 
developed with teacher and principal 
involvement; 

SFSF indicator (a)(3) 

Whether the systems used to evaluate the 
performance of teachers include student 
achievement outcomes or student growth 
data as an evaluation criterion. 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



Investing in Innovation Fund; Notice 
Inviting Applications for New Awards for 
Fiscal Year (FY) 2010, Absolute Priority 1— 
Innovations That Support Effective 
Teachers and Principals 
Under this priority, the Department 
provides funding to support practices, 
strategies, or programs that are designed 
to increase the number or percentages of 
teachers or principals who are highly 
effective teachers or principals or reduce 
the number or percentages of teachers or 
principals who are ineffective, especially 
for teachers of high-need students, by 
identifying ... highly effective teachers or 
principals (or removing ineffective 
teachers or principals). In such initiatives, 
teacher or principal effectiveness should 
be determined through an evaluation 
system that is rigorous, transparent, and 
fair; performance should be differentiated 
using multiple rating categories of 
effectiveness; multiple measures of 
effectiveness should be taken into 
account, with data on student growth as a 
significant factor. 

Final requirements for School 
Improvement Grants (2010 Notice) and 
Interventions under RTT Selection 
Criterion (E)(2): Required district activities 
under the SIG transformation model 
(i) Required activities. The LEA must— 

(B) Use rigorous, transparent, and 
equitable evaluation systems for teachers 
and principals that— 

(Note: schools implementing the 
transformation model would be required 
to implement evaluation systems that 
included student achievement gains) 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



(1) Take into account data on student 
growth as a significant factor as well as 
other factors such as multiple 
observation-based assessments of 
performance and ongoing collections of 
professional practice reflective of student 
achievement and increased high school 
graduations rates. 

Principal evaluation 
practices included 
student achievement 
gains 

Reported that its principal evaluation 
practices included: 

• Student achievement growth or gains in 
determining the principal's 
performance rating 

Source: 2011 School Survey, item 6 

Teacher Incentive Fund core element C 
(2010 Application) 

Rigorous, transparent, and fair evaluation 
systems for teachers and principals that 
differentiate effectiveness using multiple 
rating categories that take into account 
student growth (as defined in this notice) 
as a significant factor, as well as classroom 
observations conducted at least twice 
during the school year. 

RTT selection criterion (D)(2) 

The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan and ambitious yet 
achievable annual targets to ensure that 
participating LEAs— 

(ii) Design and implement rigorous, 
transparent, and fair evaluation systems 
for teachers and principals that (a) 
differentiate effectiveness using multiple 
rating categories that take into account 
data on student growth as a significant 
factor, and (b) are designed and 
developed with teacher and principal 
involvement; 

SFSF indicator (a)(6) 

Whether the systems used to evaluate the 
performance of principals include student 
achievement outcomes or student growth 
data as an evaluation criterion. 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



Investing in Innovation Fund; Notice 
Inviting Applications for New Awards for 
Fiscal Year (FY) 2010, Absolute Priority 1— 
Innovations That Support Effective 
Teachers and Principals 
Under this priority, the Department 
provides funding to support practices, 
strategies, or programs that are designed 
to increase the number or percentages of 
teachers or principals who are highly 
effective teachers or principals or reduce 
the number or percentages of teachers or 
principals who are ineffective, especially 
for teachers of high-need students, by 
identifying ... highly effective teachers or 
principals (or removing ineffective 
teachers or principals). In such initiatives, 
teacher or principal effectiveness should 
be determined through an evaluation 
system that is rigorous, transparent, and 
fair; performance should be differentiated 
using multiple rating categories of 
effectiveness; multiple measures of 
effectiveness should be taken into 
account, with data on student growth as a 
significant factor. 

Final requirements for School 
Improvement Grants (2010 Notice) and 
Interventions under RTT Selection 
Criterion (E)(2): Required district activities 
under the SIG transformation model (i) 
Required activities. The LEA must— 

(B) Use rigorous, transparent, and 
equitable evaluation systems for teachers 
and principals that— 

(1) Take into account data on student 
growth as a significant factor as well as 
other factors such as multiple 
observation-based assessments of 
performance and ongoing collections of 
professional practice reflective of 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



student achievement and increased high 
school graduations rates 
(Note: schools implementing the 
transformation model would be required 
to implement a principal evaluation 
systems that included student 
achievement gains) 

Teacher compensation 
differentiated based on 
student achievement 
gains 

Reported that its educator compensation 

practices included at least one of the 

following: 

• Teacher compensation practices 
include base pay increases, add-ons, or 
stipends to teachers based in part on 
achievement gains of students in 
individual teachers' classes 

• Teacher compensation practices 
include one-time bonuses to teachers in 
addition to base pay for achievement 
gains of students in individual teachers' 
classes 

• Teacher compensation practices 
include one-time bonuses to teachers in 
addition to base pay for achievement 
gains of students served by teacher 
grade-level or other teams 

• Teacher compensation practices 
include one-time bonuses to teachers in 
addition to base pay for average 
achievement gains of students school- 
wide 

Source: 2011 School Survey, item 7 

Teacher Incentive Fund Absolute Priority 1 
(2010 Application) 

Priority 1 (Absolute) - Differentiated 
Levels of Compensation for Effective 
Teachers and Principals: 

To meet this absolute priority, an 
applicant must demonstrate, in its 
application, that it will develop and 
implement a performance-based 
compensation system (PBCS) that 
rewards, at differentiated levels, teachers 
and principals who demonstrate their 
effectiveness by improving student 
achievement (as defined in this notice) as 
part of the coherent and integrated 
approach of the local educational agency 
(LEA) to strengthening the educator 
workforce. 

RTT selection criterion (D)(2) 

The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan and ambitious yet 
achievable annual targets to ensure that 
participating LEAs— 

(ii) Design and implement rigorous, 
transparent, and fair evaluation systems 
for teachers and principals that (a) 
differentiate effectiveness using multiple 
rating categories that take into account 
data on student growth as a significant 
factor, and (b) are designed and 
developed with teacher and principal 
involvement; 

(iv) Use these evaluations, at a minimum, 
to inform decisions regarding— 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



(b) Compensating, promoting, and 
retaining teachers and principals, 
including by providing opportunities for 
highly effective teachers and principals 
(both as defined in this notice) to obtain 
additional compensation and be given 
additional responsibilities; 

Final requirements for School 
Improvement Grants (2010 Notice) and 
Interventions under RTT Selection 
Criterion (E)(2) : Required district activities 

under the SIG transformation model 
(i) Required activities. The LEA must— 

(C) Identify and reward school leaders, 
teachers and other staff who, in 
implementing this model, have increased 
student achievement and high school 
graduation rates and identify and remove 
those who, after ample opportunities 
have been provided for them to improve 
their professional development, have not 
done so; (also a permissible activity under 
the Turnaround model) 

(Note: schools implementing the 
transformation model would be required 
to identify and reward teachers who 
increased student achievement) 

Teacher tenure, 
dismissal, or 
reassignment decisions 
used student 
achievement gains 

Reported that it used at least one of the 
following in educator tenure, assignment 
and retention decisions: 

• Use gains or growth in achievement of 
teachers' students in deciding teacher 
for tenure or dismissal 

• Use gains or growth in achievement of 
teachers' students in deciding non- 
retention with the district 

• Use gains or growth in achievement of 
teachers' students in deciding retention 
in the school or reassignment to 
another school 

Source: 2011 School Survey, item 8 

RTT Selection Criterion (D)(2)(iv)(c) and (d) 
The extent to which the State, in 
collaboration with its participating LEAs, 
has a high-quality plan and ambitious yet 
achievable annual targets to ensure that 
participating LEAs— 

(iv) use these evaluations, at a minimum, 
to inform decisions regarding (c) whether 
to grant tenure and/or full certification 
(where applicable) to teachers and 
principals using rigorous standards and 
streamlined, transparent, and fair 
procedures; and (d) removing ineffective 
tenured and untenured teachers and 
principals after they have had ample 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



opportunities to improve, and ensuring 
that such decisions are made using 
rigorous standards and streamlined, 
transparent, and fair procedures. 

SFSF Descriptors (a)(1) 

The systems used to evaluate the 
performance of teachers and the use of 
results from those systems in decisions 
regarding teacher development, 
compensation, promotion, retention, and 
removal. 

Investing in Innovation Fund; Notice 
Inviting Applications for New Awards for 
Fiscal Year (FY) 2010, Absolute Priority 1- 
Innovations That Support Effective 
Teachers and Principals 
Under this priority, the Department 
provides funding to support practices, 
strategies, or programs that are designed 
to increase the number or percentages of 
teachers or principals who are highly 
effective teachers or principals or reduce 
the number or percentages of teachers or 
principals who are ineffective, especially 
for teachers of high-need students, by 
identifying ... highly effective teachers or 
principals (or removing ineffective 
teachers or principals). In such initiatives, 
teacher or principal effectiveness should 
be determined through an evaluation 
system that is rigorous, transparent, and 
fair; performance should be differentiated 
using multiple rating categories of 
effectiveness; multiple measures of 
effectiveness should be taken into 
account, with data on student growth as a 
significant factor. 
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Table B-3. Educator workforce development indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



Final requirements for School 
Improvement Grants (2010 Notice) and 
Interventions under RTT Selection 
Criterion (E)(2): Required district activities 
under the SIG transformation model -see 
above (school requirements are not 
specified, but schools using a 
transformation model would be subject to 
district requirements). 


1 For indicators where the source is 2011 SEA survey, the text listed in the components column is the actual survey item 
wording. 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level 


Indicator 

Components 1 

Recovery Act program requirements 


SEA level 


State allowed for 

Reported one or more of the following: 

RTT selection criterion (F)(2) 

expansion of the 

• A state has no cap on the number of 

The extent to which— 

number of charter 

charter schools 

(i) The State has a charter school law that 

schools 

• A state passed legislation to increase 

does not prohibit or effectively inhibit 


the permissible number of charter 

increasing the number of high-performing 


schools 

charter schools in the State, measured by the 


• A state removed the prohibitions on 

percentage of total schools in the State that 


charter schools 

are allowed to be charter schools or 
otherwise restrict student enrollment in 


Source: National Alliance for Public 
Charter Schools annual reports (2010) 

charter schools; 

SFSF indicator (d)(7) 

The number of charter schools that are 
currently permitted to operate under State 
law. 

SFSF indicator (d)(8) 

The number of charter schools currently 
operating. 

SEA provided 

Reported providing technical assistance 

RTT selection criterion (E)(2) 

guidance on choosing 

and guidance on choosing and 

The extent to which the State has a high- 

and implementing 

implementing the four school 

quality plan and ambitious yet achievable 

school intervention 

intervention models defined by ED. 

annual targets to— 

models defined by ED 


(ii) Support its LEAs in turning around these 

Source: 2011 SEA survey, item 25 

schools by implementing one of the four 
school intervention models: turnaround 
model, restart model, school closure, or 
transformation model (provided that an LEA 
with more than nine persistently lowest- 
achieving schools may not use the 
transformation model for more than 50 
percent of its schools). 

School Improvement Grants, SEA award 
priorities 1. A. 2(a), 2(b), 2(c), 2(d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees 
to implement, and demonstrates the capacity 
to implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: Transformation model 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



a) Turnaround model 

b) Restart model 

c) School closure 

SEA supported using 
compensation 
incentives to improve 
staffing at low- 
performing schools 

Reported at least one of the following 
components in state-developed system, 
standards, guidelines, or technical 
assistance for differentiated 
compensation systems: 

• Higher starting salaries, add-ons, 
stipends, or signing bonuses for 
teachers who move to low- 
performing schools 

• Loan forgiveness or tuition support 
for teachers who move to low- 
performing schools 

• bonuses or stipends for principals 
remaining in or transferring to hard- 
to-staff or low-performing schools 

Source: 2011 SEA survey, item 20 and 
22 

RTT selection criterion (D)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan and ambitious yet achievable annual 
targets to— 

(i) Ensure the equitable distribution of teachers 
and principals by developing a plan, informed 
by reviews of prior actions and data, to ensure 
that students in high-poverty and/or high- 
minority schools have equitable access to 
highly effective teachers and principals and are 
not served by ineffective teachers and 
principals at higher rates than other students; 
and 

(ii) Increase the number and percentage of 
effective teachers teaching hard-to-staff 
subjects and specialty areas including 
mathematics, science, and special education; 
teaching in language instruction educational 
programs (as defined under Title III of the 
ESEA); and teaching in other areas as identified 
by the State or LEA. 

Plans for (i) and (ii) may include, but are not 
limited to, the implementation of incentives 
and strategies in such areas as recruitment, 
compensation, teaching and learning 
environments, professional development, and 
human resources practices and processes. 

SEA monitored 
deployment of 
effective educators in 
low-performing 
schools 

Reported using data from local 
educator evaluation systems to monitor 
the deployment of effective educators 
in low-performing schools 

Source: 2011 SEA survey, item 25 

RTT selection criterion (D)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan and ambitious yet achievable annual 
targets to— 

(i) Ensure the equitable distribution of teachers 
and principals by developing a plan, informed 
by reviews of prior actions and data, to ensure 
that students in high-poverty and/or high- 
minority schools have equitable access to 
highly effective teachers and 


(continued) 


B-44 


Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



principals and are not served by ineffective 
teachers and principals at higher rates than 
other students. 

District level 

District targeted low- 
performing schools 
for closure 2 

Reported using the following strategy 
to support school restructuring: 

• Targeting individual chronically low- 
performing school for closure 

Source: 2011 District survey, item 9 

School Improvement Grants, SEA award 
priorities 1. A. 2(c) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(c) SIG School Closure model 
School closure occurs when an LEA closes a 
school and enrolls the students who attended 
that school in other schools in the LEA that are 
higher achieving. These other schools should be 
within reasonable proximity to the closed 
school and may include, but need not be 
limited to, charter schools or new schools for 
which achievement data are not yet available. 

District replaced 
principal and 
teachers in low- 
performing schools 2 

Reported using both of the following 
strategies to support school 
restructuring: 

• Replacing a substantial proportion 
of the teachers in individual low- 
performing schools 

• Replacing principals in individual 
low- performing schools 

Source: 2011 District survey, item 9 

School Improvement Grants, SEA award 
priorities 1. A. 2(a) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an LEA 

must- 

(i) Replace the principal and grant the new 
principal sufficient operational flexibility 
(including flexibility in staffing, scheduling, and 
budgeting) to implement a comprehensive 
approach in order to substantially improve 
student achievement outcomes and increase 
high school graduation rates; 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



(ii) Use locally adopted competencies to 
measure the effectiveness of staff who can 
work within the turnaround environment to 
meet the needs of students, 

(A) Screen all existing staff and rehire no more 
than 50 percent; and 

(B) Select new staff; 

District contracted 
with external 
organization to 
operate low- 
performing schools 2 

Reported using the following strategy 
to support school restructuring: 

• Contracting with CMOs, EMOs, or 
community organizations to 
operate schools, and targeting the 
strategy to low-performing schools 

Source: 2011 District survey, item 9 

School Improvement Grants, SEA award 
priorities 1. A. 2(b) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(b) SIG Restart model 

A restart model is one in which an LEA converts 
a school or closes and reopens a school under a 
charter school operator, a charter management 
organization (CMO), or an education 
management organization (EMO) that has been 
selected through a rigorous review process. (A 
CMO is a non-profit organization that operates 
or manages charter schools by centralizing or 
sharing certain functions and resources among 
schools. An EMO is a for-profit or non-profit 
organization that provides "whole-school 
operation" services to an LEA.) A restart model 
must enroll, within the grades it serves, any 
former student who wishes to attend the 
school. 

District provided 
compensation 
incentives to improve 
staffing at low- 
performing schools 2 

Reported that its educator 
compensation system included at least 
one of the following: 

• Higher starting salaries, add-ons, 
stipends, or signing bonuses for 
teachers who move to low- 
performing schools 

• Loan forgiveness or tuition support 
for teachers who move to low- 
performing schools 

Source: 2011 District survey, item 6 

RTT selection criterion (D)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan and ambitious yet achievable annual 
targets to— 

(i) Ensure the equitable distribution of teachers 
and principals by developing a plan, informed 
by reviews of prior actions and data, to ensure 
that students in high-poverty and/or high- 
minority schools have equitable access to 
highly effective teachers and principals and are 
not served by ineffective 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



teachers and principals at higher rates than 
other students; and 

(ii) Increase the number and percentage of 
effective teachers teaching hard-to-staff 
subjects and specialty areas including 
mathematics, science, and special education; 
teaching in language instruction educational 
programs (as defined under Title III of the 
ESEA); and teaching in other areas as identified 
by the State or LEA. 

Plans for (i) and (ii) may include, but are not 
limited to, the implementation of incentives 
and strategies in such areas as recruitment, 
compensation, teaching and learning 
environments, professional development, and 
human resources practices and processes. 

School Improvement Grants, SEA award 
priorities 1. A. 2(a) and (d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an LEA 

must- 

(iii) Implement such strategies as financial 
incentives, increased opportunities for 
promotion and career growth, and more 
flexible work conditions that are designed to 
recruit, place, and retain staff with the skills 
necessary to meet the needs of the students in 
the turnaround school; 

(d) SIG Transformation model 
(l)(i)Required activities. The LEA must— 

(E) Implement such strategies as financial 
incentives, increased opportunities for 
promotion or career growth, and more flexible 
work conditions that are designed to recruit, 
place, and retain staff with the skills necessary 
to meet the needs of the students in a 
transformation school. 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



(l)(ii) Permissible activities. An LEA may also 
implement other strategies to develop 
teachers' and school leaders' effectiveness, 
such as- 

(A) Providing additional compensation to 
attract and retain staff with the skills necessary 
to meet the needs of the students in a 
transformation school; 

District extended 
school day, week, or 
year in low- 
performing schools 2 

Reported using at least one of the 
following strategies to support school 
restructuring: 

• Extending the regular school day 
and/or week, including "Saturday" 
school or before/after school 
sessions (required for some 
students), and targeting the 
strategy to low-performing schools 

• Extending the regular school year, 
and targeting the strategy to low- 
performing schools 

Source: 2011 District survey, item 9 

School Improvement Grants, SEA award 
priorities 1. A. 2(a) and (d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an LEA 

must- 

(viii) Establish schedules and implement 
strategies that provide increased learning time 
(as defined in this notice)*; 

(d) SIG Transformation model 

(3)(i) Required activities. The LEA must- 

(A) Establish schedules and strategies that 
provide increased learning time (as defined in 
this notice)*; 

(ii) Permissible activities. An LEA may also 
implement other activities that extend learning 
time and create community oriented schools, 
such as— 

(B) Extending or restructuring the school day so 
as to add time for such strategies as advisory 
periods that build relationships between 
students, faculty, and other school staff; 

increased learning time means using a longer 
school day, week, or year schedule to 
significantly increase the total number of 
school hours... 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 

District required low- 
performing schools 
to partner with 
organizations that 
specialize in 
instructional 
improvement 2 

Reported using the following strategy 
to help individual schools improve 
instruction and related support 
activities: 

• Requiring schools to partner with 
an organization that specializes in 
instructional improvement (e.g., 
local universities or outside 
vendors) and targeted the strategy 
to low-performing schools 

Source: 2011 District survey, item 10 

School Improvement Grants, SEA award 
priorities 1. A. 2(d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(d) SIG Transformation model 
(4)(i) Required activities. The LEA must- 
(B) Ensure that the school receives ongoing, 
intensive technical assistance and related 
support from the LEA, the SEA, or a designated 
external lead partner organization (such as a 
school turnaround organization or an EMO). 

District implemented 
programs in low- 
performing schools 
to encourage family 
and community 
involvement 2 

Reported using the following strategy 
to help individual schools improve 
instruction and related support 
activities: 

• Implementing school-level 

programs to encourage family and 
community involvement, and 
targeting the strategy to low- 
performing schools 

Source: 2011 District survey, item 10 

School Improvement Grants, SEA award 
priorities 1. A. 2(d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(d) SIG Transformation model 

(3)(i) Required activities. The LEA must- 

(B) Provide ongoing mechanisms for family and 

community engagement; 

(ii) Permissible activities. An LEA may also 
implement other strategies that extend 
learning time and create community-oriented 
schools, such as— 

(A) Partnering with parents and parent 
organizations, faith- and community-based 
organizations, health clinics, other State or 
local agencies, and others to create safe school 
environments that meet students' social, 
emotional, and health needs; 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 

District provided 
school leaders in 
low-performing 
schools with staffing 
or budgeting 
flexibility to 
implement school 
reform 2 

Reported using the following strategy 
to support school restructuring: 

• Allowing school leaders to 
deviate from standard district 
staffing or budgeting patterns to 
implement school-specific 
reforms, and targeting the 
strategy to low-performing 
schools 

Source: 2011 District survey, item 9 

School Improvement Grants, SEA award 
priorities 1. A. 2(a) and (d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees 
to implement, and demonstrates the capacity 
to implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an 

LEA must— 

(i) Replace the principal and grant the 
principal sufficient operational flexibility 
(including in staffing, calendars/time, and 
budgeting) to implement fully a 
comprehensive approach in order to 
substantially improve student achievement 
outcomes and increase high school 
graduation rates. 

(d) SIG Transformation model 
(l)(ii) Permissible activities. An LEA may also 
implement other strategies to develop 
teachers' and school leaders' effectiveness, 
such as— 

(C) Ensuring that the school is not required to 
accept a teacher without the mutual consent 
of the teacher and principal, regardless of the 
teacher's seniority. 

(4)(i) Required activities. The LEA must— 

(A) Give the school sufficient operational 
flexibility (such as staffing, calendars, and 
budgeting) to implement fully a 
comprehensive approach to substantially 
improve student achievement outcomes and 
increase high school graduation rates; 

District provided 
technical assistance 
to low-performing 
schools to screen or 
select school 
improvement 
experts or models 2 

Reported using at least one of the 
following strategies to support school 
restructuring: 

• Providing technical assistance to 
help schools Identify and screen 
potential school improvement 
models, and targeted the strategy 
to low-performing schools 

School Improvement Grants, SEA award 
priorities 1. A. 2(a) and (d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees 
to implement, and demonstrates the capacity 
to implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 


• Providing technical assistance to 
help schools screen and select school 
improvement experts for low- 
performing schools, and targeted the 
strategy to low-performing schools 

Source: 2011 District survey, item 9 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an 

LEA must- 

(iv) Provide staff ongoing, high-quality, job- 
embedded professional development that is 
aligned with the school's comprehensive 
instructional program and designed with 
school staff to ensure that they are equipped 
to facilitate effective teaching and learning 
and have the capacity to successfully 
implement school reform strategies; 

(vi) Use data to identify and implement an 
instructional program that is research-based 
and vertically aligned from one grade to the 
next as well as aligned with State academic 
standards; 

(d) SIG Transformation model 

(l)(i) Required activities. The LEA must— 

(D) Provide staff ongoing, high quality, job- 
embedded professional development (e.g., 
regarding subject specific 
pedagogy, instruction that reflects a deeper 
understanding of the community served by 
the school, or differentiated instruction) that 
is aligned with the school's comprehensive 
instructional program and designed with 
school staff to ensure they are equipped to 
facilitate effective teaching and learning and 
have the capacity to successfully implement 
school reform 
strategies; 

(4)(i)(B) Ensure that the school receives 
ongoing, intensive technical assistance and 
related support from the LEA, SEA, or 
designated external lead partner organization 
(such as a school turnaround organization or 
an EMO). 

School level 

Staffing or budgeting 
autonomy increased 
as part of school 
restructuring 3 

Reported introducing the following 
features of school restructuring or 
reorganization at the school: 

• Increased autonomy in staffing and 
budgeting from the district, CMO or 
EMO, compared to before 

School Improvement Grants, SEA award 
priorities 1. A. 2(a) and (d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees 
to implement, and demonstrates the capacity 
to implement fully and effectively. 


(continued) 


B-51 


Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 


restructuring or reorganizing 
Source: 2011 School survey, item 10 

one of the following rigorous interventions in 
each Tier 1 and Tier II school that the LEA 
commits to serve: 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an LEA 

must- 

(i) Replace the principal and grant the principal 
sufficient operational flexibility (including in 
staffing, calendars/time, and budgeting) to 
implement fully a comprehensive approach in 
order to substantially improve student 
achievement outcomes and increase high 
school graduation rates. 

(d) SIG Transformation model 
(l)(ii) Permissible activities. An LEA may also 
implement other strategies to develop 
teachers' and school leaders' effectiveness, 
such as— 

(C) Ensuring that the school is not required to 
accept a teacher without the mutual consent of 
the teacher and principal, regardless of the 
teacher's seniority. 

(4)(i) Required activities. The LEA must— 

(A) Give the school sufficient operational 
flexibility (such as staffing, calendars, and 
budgeting) to implement fully a comprehensive 
approach to substantially improve student 
achievement outcomes and increase high 
school graduation rates; 

School day, week, or 
year extended as part 
of school 
restructuring 3 

Reported introducing at least one the 
following features of school 
restructuring or reorganization at the 
school: 

• Extended regular school day 
and/or week 

• Extended regular school year 
Source: 2011 School survey, item 10 

School Improvement Grants, SEA award 
priorities 1. A. 2(a) and (d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an LEA 

must- 

(viii) Establish schedules and implement 
strategies that provide increased learning time 
(as defined in this notice)*; 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



(d) SIG Transformation model 

(3)(i) Required activities. The LEA must- 

(A) Establish schedules and strategies that 
provide increased learning time (as defined in 
this notice)*; 

(ii) Permissible activities. An LEA may also 
implement other activities that extend learning 
time and create community oriented schools, 
such as— 

(B) Extending or restructuring the school day so 
as to add time for such strategies as advisory 
periods that build relationships between 
students, faculty, and other school staff; 

increased learning time means using a longer 
school day, week, or year schedule to 
significantly increase the total number of 
school hours... 

School modified daily 
schedule to increase 
instructional time for 
reading/English 
language arts or 
mathematics 3 

Reported using the following strategy 
to help improve instruction and related 
student services: 

• Modified their daily schedule to 
increase the amount of 
instructional time for 
reading/English language arts or 
mathematics. 

Source: 2011 School survey, item 11 

School Improvement Grants, SEA award 
priorities 1. A. 2(d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an LEA 

must- 

(viii) Establish schedules and implement 
strategies that provide increased learning time 
(as defined in this notice)*; 

(d) SIG Transformation model 
(3)(i) Required activities. The LEA must- 
(A) Establish schedules and strategies that 
provide increased learning time (as defined in 
this notice)*; 

(ii) Permissible activities. An LEA may also 
implement other activities that extend learning 
time and create community oriented schools, 
such as— 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



(B) Extending or restructuring the school day 
so as to add time for such strategies as advisory 
periods that build relationships between 
students, faculty, and other school staff; 

■“Increased learning time means using a longer 
school day, week, or year schedule to 
significantly increase the total number of 
school hours... 

Compensation 
incentives to 
encourage teachers 
to move to or remain 
in the school 3 

Reported that its compensation 
practices included: 

• Higher starting salaries, add-ons, 
stipends, or bonuses for teachers 
who move to teach in the school 

Source: 2011 School survey, item 7 

RTT selection criterion (D)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan and ambitious yet achievable annual 
targets to— 

(i) Ensure the equitable distribution of teachers 
and principals by developing a plan, informed 
by reviews of prior actions and data, to ensure 
that students in high-poverty and/or high- 
minority schools have equitable access to 
highly effective teachers and principals and are 
not served by ineffective teachers and 
principals at higher rates than other students; 
and 

(ii) Increase the number and percentage of 
effective teachers teaching hard-to-staff 
subjects and specialty areas including 
mathematics, science, and special education; 
teaching in language instruction educational 
programs (as defined under Title III of the 
ESEA); and teaching in other areas as identified 
by the State or LEA. 

Plans for (i) and (ii) may include, but are not 
limited to, the implementation of incentives 
and strategies in such areas as recruitment, 
compensation, teaching and learning 
environments, professional development, and 
human resources practices and processes. 

School Improvement Grants, SEA award 
priorities 1. A. 2(a) and (d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
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Table B-4. SEA, district, and school indicators, components, and Recovery Act program requirements 
for supporting improvement for low-performing schools (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an LEA 

must- 

(iii) Implement such strategies as financial 
incentives, increased opportunities for 
promotion and career growth, and more 
flexible work conditions that are designed to 
recruit, place, and retain staff with the skills 
necessary to meet the needs of the students in 
the turnaround school; 

(d) SIG Transformation model 
(l)(i)Required activities. The LEA must— 

(E) Implement such strategies as financial 
incentives, increased opportunities for 
promotion or career growth, and more flexible 
work conditions that are designed to recruit, 
place, and retain staff with the skills necessary 
to meet the needs of the students in a 
transformation school. 

(l)(ii) Permissible activities. An LEA may also 
implement other strategies to develop 
teachers' and school leaders' effectiveness, 
such as- 

(A) Providing additional compensation to 
attract and retain staff with the skills necessary 
to meet the needs of the students in a 
transformation school; 

Nonfinancial 
incentives to 
encourage teachers 
to move to or remain 
in the school 3 

Reported that its compensation 
practices included: 

• Non-financial incentives (e.g., 
smaller class size, planning time, 
reduced classroom hours) to 
encourage teachers to come to and 
remain in the school. 

Source: 2011 School survey, item 7 

School Improvement Grants, SEA award 
priorities 1. A. 2(a) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an LEA 

must— 

(iii) Implement such strategies as financial 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



incentives, increased opportunities for 
promotion and career growth, and more 
flexible work conditions that are designed to 
recruit, place, and retain staff with the skills 
necessary to meet the needs of the students in 
the turnaround school; 

Substantial portion of 
teachers replaced as 
part of school 
restructuring 3 

Reported that it introduced the 
following feature of school 
restructuring or reorganization at the 
school: 

• Replacement of a substantial 

proportion of teachers introduced 
as a feature of school restructuring 
or reorganization. 

Source: 2011 School survey, item 10 

School Improvement Grants, SEA award 
priorities 1. A. 2(a) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an LEA 

must- 

(ii) Use locally adopted competencies to 
measure the effectiveness of staff who can 
work within the turnaround environment to 
meet the needs of students, 

(A) Screen all existing staff and rehire no more 
than 50 percent; and 

(B) Select new staff; 

Effective teachers 
reassigned to school 
as part of school 
restructuring 3 

Features of school restructuring or 
reorganization introduced at the school 
included: 

• Reassignment of effective teachers 
to the school from others 

Source: 2011 School survey, item 10 

RTT selection criterion (D)(3) 

The extent to which the State, in collaboration 
with its participating LEAs, has a high-quality 
plan and ambitious yet achievable annual 
targets to— 

(i) Ensure the equitable distribution of teachers 
and principals by developing a plan, informed 
by reviews of prior actions and data, to ensure 
that students in high-poverty and/or high- 
minority schools have equitable access to 
highly effective teachers and principals and are 
not served by ineffective teachers and 
principals at higher rates than other students; 
and 

(ii) Increase the number and percentage of 
effective teachers teaching hard-to-staff 
subjects and specialty areas including 
mathematics, science, and special education; 
teaching in language instruction educational 
programs (as defined under Title III of the 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



ESEA); and teaching in other areas as identified 
by the State or LEA. 

Plans for (i) and (ii) may include, but are not 
limited to, the implementation of incentives 
and strategies in such areas as recruitment, 
compensation, teaching and learning 
environments, professional development, and 
human resources practices and processes. 

School Improvement Grants, SEA award 
priorities 1. A. 2(a) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(a) SIG Turnaround model 

(1) A turnaround model is one in which an LEA 

must- 

(iii) Implement such strategies as financial 
incentives, increased opportunities for 
promotion and career growth, and more 
flexible work conditions that are designed to 
recruit, place, and retain staff with the skills 
necessary to meet the needs of the students in 
the turnaround school; 

School Improvement Grants, SEA award 
priorities 1. A. 2(d) 

SIG Transformation model 
(l)(i)Required activities. The LEA must— 

(E) Implement such strategies as financial 
incentives, increased opportunities for 
promotion or career growth, and more flexible 
work conditions that are designed to recruit, 
place, and retain staff with the skills necessary 
to meet the needs of the students in a 
transformation school. 

(l)(ii) Permissible activities. An LEA may also 
implement other strategies to develop 
teachers' and school leaders' effectiveness, 
such as- 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 



A) Providing additional compensation to attract 
and retain staff with the skills necessary to 
meet the needs of the students in a 
transformation school; 

School implemented 

Reported that it used the following 

School Improvement Grants, SEA award 

programs to 

strategy to help improve instruction 

priorities 1. A. 2(d) 

encourage family and 

and related student services: 

Strongest Commitment. An LEA with the 

community 

• Implemented programs to 

strongest commitment is an LEA that agrees to 

involvement 3 

encourage family and community 
involvement 

Source: 2011 School survey, item 11 

implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(d) SIG Transformation model 

(3)(i) Required activities. The LEA must- 

(B) Provide ongoing mechanisms for family and 

community engagement; 

School implemented 

Reported that it used the following 

School Improvement Grants, SEA award 

programs to orient 

strategy to help improve instruction 

priorities 1. A. 2(d) 

parents to school 

and related student services: 

Strongest Commitment. An LEA with the 

improvement 

• Implemented programs to orient 

strongest commitment is an LEA that agrees to 

models 3 

parents to school improvement 
models 

Source: 2011 School survey, item 11 

implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(d) SIG Transformation model 

(3)(i) Required activities. The LEA must- 

(B) Provide ongoing mechanisms for family and 

community engagement; 

School implemented 

Reported that it used the following 

School Improvement Grants, SEA award 

programs to address 

strategy to help improve instruction 

priorities 1. A. 2(d) 

students' social and 

and related student services: 

SIG Transformation model 

emotional needs 3 

• Implemented programs to address 
students' social and emotional 
needs 

Source: 2011 School survey, item 11 

(3)(ii) Permissible activities. An LEA may also 
implement other strategies that extend 
learning time and create community-oriented 
schools, such as— 

(A)Partnering with parents and parent 
organizations, faith- and community-based 
organizations, health clinics, other State or 
local agencies, and others to create safe school 
environments that meet students' social, 
emotional, and health needs; 
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Table B-4. Support for low-performing schools indicators, components, and Recovery Act program 
requirements, by education level (cont'd) 


Indicator 

Components 1 

Recovery Act program requirements 

School used outside 
school improvement 
experts as part of 
school restructuring 
or to improve 
instruction 3 

Reported that it introduced the 
following feature of school 
restructuring or reorganization 
introduced at the school included: 

• Used school improvement experts 
from outside the school introduced 
as part of school restructuring or 
reorganizing 

or 

Reported using the following strategy 
to help improve instruction and related 
student services: 

• Used a school improvement model 
developed by and outside partner 
or vendor to improve instruction 
and related services 

Source: 2011 School survey, items 10 
and 11 

School Improvement Grants, SEA award 
priorities 1. A. 2(d) 

Strongest Commitment. An LEA with the 
strongest commitment is an LEA that agrees to 
implement, and demonstrates the capacity to 
implement fully and effectively, one of the 
following rigorous interventions in each Tier 1 
and Tier II school that the LEA commits to 
serve: 

(d) SIG Transformation model 
(4)(i) Required activities. The LEA must- 
(B) Ensure that the school receives ongoing, 
intensive technical assistance and related 
support from the LEA, the SEA or a designated 
external lead partner organization (such as a 
school turnaround organization or an EMO). 

(ii) Permissible activities The LEA may also 
implement other strategies for providing 
operational flexibility and intensive support, 
such as— 

(A) Allowing the school to be run under a new 
governance arrangement, such as a turnaround 
division within the LEA or SEA; 


1 For indicators where the source is 2011 SEA survey, the text listed in the components column is the actual survey item 
wording. 

2 Indicator limited to districts with low-performing schools. 

3 Indicator limited to low-performing schools. 
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Table C-l. Confidence intervals for percentage of districts in Common Core states that implemented 
reforms related to new or revised state standards: 2010-11 and 2011-12 




2010-11 



2011-12 




Low 

High 


Low 

High 

Indicator description 

% 

Cl 

Cl 

% 

Cl 

Cl 

District Aware of State Adoption of Common Core State 
Standards 

86 

79.0 

93.8 

§ 

98 

96.7 

99.0 

District Provided Professional Development on New or Revised 







Standards on Instructional Strategies for Educators Who Teach or 
Mentor Math or English Language Arts 

59 

51.0 

66.2 

§ 

73 

67.2 

79.5 

District Provided Professional Development on New or Revised 
Standards on Instructional Strategies for Teachers to Help ELL or 
Students with Disabilities Master the Standards 

58 

50.3 

65.7 

58 

51.1 

65.0 

District Distributed Instructional Materials or Provided Selection 
Guidance on Curricula Aligned with the New or Revised State 
Content Standards 

53 

45.8 

59.9 

§ 

63 

55.5 

70.4 


§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < 0.05). 

Notes: The percentages in the table are cross-sectional estimates for the population of districts in CCSS states that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this 
table, for each year, the denominator is the estimated number of districts in CCSS states as of that year. No data are reported 
for 2009-10 because the Common Core State Standards were not yet available. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 District Surveys. 
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Table C-2. Results of significance tests comparing percentages of districts in Common Core states 
that implemented reforms related to new or revised state standards: 2011-12 


Ref# 

Indicator description 

Percent 
of districts 1 

1 

District Aware of State Adoption of Common Core State Standards 

98 

2 

District Provided Professional Development on New or Revised State Content Standards for 
Educators Who Teach or Mentor Mathematics or Reading/English Language Arts 

73 

3 

District Provided Professional Development on New or Revised State Content Standards on 
Instructional Strategies for Teachers to Help English Learners or Students with Disabilities 
Master the Content Standards 

58 

4 

District Distributed Instructional Materials or Provided Selection Guidance on Curricula 
Aligned with the New or Revised State Content Standards 

63 


1 These percentages show the percentage of districts in CCSS states that satisfied the requirements of the indicator. They are 
cross-sectional estimates for the population of districts operating in the 2010-11 school year. See appendix A for more 
information about the generalizability of these estimates. 



^Percentages are significantly different (p < 0.05). 

Table reads: A significantly larger percentage of districts were aware of their state's adoption of Common Core State Standards 
(98 percent) than provided professional development on new or revised state content standards for educators who teach or 
mentor mathematics or reading/English language arts (73 percent). Use the reference number (Ref #) to identify the indicator. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table C-3. Confidence intervals for percentage of high-poverty and other districts in Common Core 
states that implemented reforms related to new or revised state content standards: 
2011-12 



High Poverty 



Other 


Indicator description 

% 

Low Cl 

High Cl 

% 

Low Cl 

High Cl 

District Aware of State Adoption of Common Core 
State Standards 

94 

90.3 

97.7 

99* 

98.6 

99.9 

District Provided Professional Development on New 
or Revised State Content Standards on Instructional 
Strategies for Educators Who Teach or Mentor 
Mathematics or Reading/English Language Arts 

64 

52.5 

74.8 

77 

69.8 

84.1 







District Provided Professional Development on New 
or Revised State Content Standards on Instructional 
Strategies for Teachers to Help English Learners or 
Students with Disabilities Master the Content 
Standards 

55 

45.5 

64.2 

59 

50.3 

68.1 

District Distributed Instructional Materials or 
Provided Selection Guidance on Curricula Aligned 
with the New or Revised State Content Standards 

59 

48.6 

69.9 

64 

55.1 

73.5 


*Percentage is significantly different from percentage for high-poverty districts (p < 0.05). 

Notes: The percentages in the table are cross-sectional estimates for the population of districts in CCSS states that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this 
table, the denominator for the high-poverty percentages is the estimated number of high-poverty districts (had a child poverty 
rate above 21.66 percent) that were in states that adopted the Common Core as of the 2011-12 school year and have sufficient 
data. The denominator for the other percentages is the estimated number of districts with sufficient data that had that had a 
child poverty rate at or below 21.66 percent that are in states that adopted the Common Core in at least one subject as of that 
year. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. U.S. Census Bureau. Small Area Income and Poverty 
Estimates (SAIPE) program, district data for 2008 (USSD08.xls) for poverty data. Retrieved November 11, 2010, from 
http://www.census.gov/did/www/saipe/data/schools/index.html . 
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Table C-4. Confidence intervals for percentage of large districts and districts of other sizes in 

Common Core states that implemented reforms related to new or revised state content 
standards: 2011-12 



Large 



Medium 



Small 


Indicator description 

% Low Cl 

High Cl 

% 

Low Cl 

High Cl 

% 

Low Cl 

High Cl 

District Aware of State Adoption 
of Common Core State 
Standards 

100 100.0 

100.0 

CD 

00 

* 

97.2 

99.6 

98* 

96.6 

99.0 

District Provided Professional 
Development on New or Revised 
State Content Standards on 
Instructional Strategies for 

91 90.5 

90.5 

75* 

70.2 

80.1 

73* 

66.8 

79.6 


Educators Who Teach or Mentor 
Mathematics or Reading/English 
Language Arts 


District Provided Professional 
Development on New or Revised 
State Content Standards on 
Instructional Strategies for 
Teachers to Help English 
Learners or Students with 
Disabilities Master the Content 
Standards 


74 74.4 74.4 61* 54.6 66.7 58* 50.6 65.1 


District Distributed Instructional 
Materials or Provided Selection 

Guidance on Curricula Aligned 73 72.6 72.6 57* 50.6 62.6 63* 55.4 70.8 

with the New or Revised State 
Content Standards 


* Percentage is significantly different from percentage for large districts (p<0.05). 

Notes: The percentages in the table are cross-sectional estimates for the population of districts in CCSS states that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this 
table, the denominator for a size group is the estimated number of districts of a given size that have sufficient data and are in 
states that adopted the Common Core in at least one subject as of the 2011-12 school year. District size is based on the student 
enrollment in schools in the district. Large districts have at least 50,000 students. Medium districts have less than 50,000 but at 
least 15,000 students. Small districts have fewer than 15,000 students. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. National Center for Education Statistics, Common Core of 
Data. Local Education Agency Universe Survey: School Year 2008-09 (ag081a.sas7bdat) for enrollment size. Retrieved August 
24, 2010, from http://nces.ed.gov/ccd/pubagency.asp . 
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Table C-5. Confidence intervals for percentage of districts in Common Core states reporting major 
challenges when planning or implementing new or revised state standards and aligned 
assessments: 2011-12 



District reported 
challenge as a major 
challenge 1 

Districts rated the 
challenge 2 

District challenge 

Percent 

Low Cl 

High Cl 

Percent 

Low Cl 

High Cl 

Insufficient funding to purchase new instructional 
materials aligned with new standards 

60 

53.4 

67.0 

95 

92.2 

97.4 

Current assessments are not aligned with the new 
standards 

57 

48.9 

65.0 

95 

93.6 

96.8 

Insufficient funding to support instructional specialists or 
coaches to help educators implement new standards 

52 

45.4 

57.8 

93 

90.2 

96.0 

Lack of district staff capacity or expertise to develop new 
curricula guides and instructional materials aligned with 
new standards 

37 

29.4 

44.6 

96 

94.7 

97.7 

Inadequate quality or availability of state-developed 
instructional materials aligned with standards 

36 

26.4 

46.5 

94 

91.7 

95.4 

Insufficient funding to provide adequate training to 
teachers on the content and use of the standards 

36 

29.2 

43.0 

96 

94.1 

97.4 

Standardized assessments not available for enough 
subjects or grades 

25 

18.0 

32.3 

96 

94.4 

97.6 

Lack of district staff capacity or expertise to provide 
guidance about or train educators on using new 
standards for their instruction 

25 

19.4 

30.9 

96 

94.8 

97.8 

Concerns or opposition from school staff about additional 
assessments 

22 

13.7 

30.5 

94 

88.1 

100.0 

Lack of clear SEA guidance or support on expectations 
concerning when and how standards should be 
implemented 

21 

13.8 

27.9 

94 

91.2 

96.0 

Lack of clear SEA guidance or support on specific content 
of new standards 

19 

10.2 

27.8 

93 

90.6 

95.5 

Lack of district staff capacity or expertise to provide 
guidance about or train educators on how to administer 
assessments 

12 

9.3 

15.7 

98 

97.1 

99.3 

Concerns or opposition from school staff or staff unions 
to new standards 

11 

7.7 

14.7 

87 

81.4 

93.5 

Concerns or opposition from parents or other community 
groups to additional assessments 

8 

4.1 

12.4 

91 

84.7 

97.3 

Concerns or opposition from parents or other community 
groups to new standards 

7 

3.7 

10.1 

88 

82.3 

94.6 


1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Districts rating the challenge excludes districts that identified the challenge as not applicable, districts that did not respond to 
the question, and districts in states that did not adopt the CCSS in both reading/English language arts and mathematics. 

Notes: The percentages in the table are cross-sectional estimates for the population of districts in CCSS states that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table C-6. Results of significance tests comparing percentages of districts in Common Core states 
reporting major challenges when planning or implementing new or revised state 
standards and aligned assessments: 2011-12 

Ref. # Challenge description Percent 1 

1 Insufficient funding to purchase new instructional materials aligned with new standards 60 

2 Current assessments are not aligned with the new standards 57 

3 Insufficient funding to support instructional specialists or coaches to help educators implement ^ 

new standards 

4 Lack of district staff capacity or expertise to develop new curricula guides and instructional ^ 

materials aligned with new standards 

5 Inadequate quality or availability of state-developed instructional materials aligned with ^ 

standards 

6 Insufficient funding to provide adequate training to teachers on the content and use of the ^ 

standards 

7 Standardized assessments not available for enough subjects or grades 25 

8 Lack of district staff capacity or expertise to provide guidance about or train educators on using ^ 

new standards for their instruction 

9 Concerns or opposition from school staff about additional assessments 22 

10 Lack of clear SEA guidance or support on expectations concerning when and how standards 
should be implemented 

11 Lack of clear SEA guidance or support on specific content of new standards 19 

12 Lack of district staff capacity or expertise to provide guidance about or train educators on how 
to administer assessments 

13 Concerns or opposition from school staff or staff unions to new standards 11 

14 Concerns or opposition from parents or other community groups to additional assessments 8 

15 Concerns or opposition from parents or other community groups to new standards 7 

1 The percentage of districts in CCSS states that rated the challenge as major. They are cross-sectional estimates for the 

population of districts operating in the 2010-11 school year. See appendix A for more information about the generalizability 

of the estimates 

(continued) 
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Table C-6. Results of significance tests comparing percentages of districts in Common Core states 
reporting major challenges when planning or implementing new or revised state 
standards and aligned assessments: 2011-12 (cont'd) 



^Percentages are significantly different (p < 0.05) 

Table reads: As denoted by no asterisk in row 1, column 2, the percentage of districts that rated insufficient funding to purchase 
new instructional materials as a major challenge (60 percent) and the percentage that rated current assessments are not 
aligned with the new standards as a major challenge (57 percent) were not significantly different. Use the reference number 
(Ref. #) to identify the challenge. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table C-7. Confidence intervals for percentage of schools in Common Core states that implemented 
new or revised state standards: 2010-11 and 2011-12 




2010-11 



2011-12 


Indicator description 

% 

Low 

Cl 

High 

Cl 

% 

Low 

Cl 

High 

Cl 

Teachers Received Professional Development on New or 
Revised State Content Standards 

63 

60.4 

65.9 

i <r 

00 

rv 

75.3 

80.5 

Teachers Received Professional Development Targeted to 
Help English Learners or Students with Disabilities Master 
New or Revised State Content Standards 

62 

58.5 

64.5 

<T» 

00 

«yi 

63.6 

71.8 

School Used Curriculum or Curriculum Materials Aligned 
with New or Revised State Content Standards 

60 

56.8 

62.4 

§ 

66 

62.7 

69.2 


§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < 0.05). 

Notes: The percentages in the table are cross-sectional estimates for the population of schools in CCSS states that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this 
table, for each year, the denominator is the estimated number of schools in the states that adopted the Common Core in at 
least one subject as of that year. No data are reported for 2009-10 because the Common Core State Standards were not yet 
available. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. 
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Table C-8. Results of significance tests comparing percentages of schools in Common Core states 
that implemented new or revised state standards: 2011-12 


Ref# 

Indicator description 

Percent of 
schools 1 

1 

Teachers Received Professional Development on New or Revised State Content 
Standards 

78 

2 

Teachers Received Professional Development Targeted to Help English Learners or 
Students with Disabilities Master New or Revised State Content Standards 

68 

3 

School Used Curriculum or Curriculum Materials Aligned with New or Revised State 
Content Standards 

66 


1 These percentages show the percentage of schools in CCSS states that satisfied the requirements of the indicator. They are 
cross-sectional estimates for the population of schools operating in the 2010-11 school year. See appendix A for more 
information about the generalizability of the estimates. 


Ref# 

1 

2 

3 

1 


* 

* 

2 




3 





*Percentages are significantly different (p < 0.05). 

Table reads: As denoted by the asterisk in row 1, column 2, a significantly larger percentage of schools in CCSS states reported 
that teachers received professional development on new or revised state content standards (78 percent) than received 
professional development targeted to help English learners or students with disabilities master new or revised state content 
standards (68 percent). Use the reference number (Ref #) to identify the indicator. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table C-9. Confidence intervals for percentage of low-performing and other schools in Common 
Core states that implemented new or revised state standards: 2011-12 



Low-performing 






schools 



Other schools 



Low 

High 


Low 

High 

Indicator description 

% 

Cl 

Cl 

% 

Cl 

Cl 

Teachers Received Professional Development on New or 
Revised State Content Standards 

73 

67.3 

78.5 

* 

79 

76.1 

81.7 

Teachers Received Professional Development Targeted to Help 
English Learners or Students with Disabilities Master New or 
Revised State Content Standards 

68 

63.5 

73.4 

68 

62.7 

72.4 

School Used Curriculum or Curriculum Materials Aligned with 
New or Revised State Content Standards 

59 

52.7 

66.3 

67* 

63.9 

70.4 


^Percentage is significantly different from percentage for low-performing schools (p < 0.05). 

Notes: The percentages in the table are cross-sectional estimates for the population of schools in CCSS states that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this 
table, the denominator is the estimated number of schools in the performance status category (low-performing, other) with 
sufficient data and in states that adopted the Common Core in at least one subject as of the 2011-12 school year. Low- 
performing schools include schools that were (1) in improvement, corrective action, or restructuring, (2) identified as among 
the lowest achieving schools, or (3) that have had a graduation rate below 60 percent over a number of years. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. Approved state applications for School Improvement grants 
for school performance status. Retrieved December 2010 from http://www2.ed.gov/programs/sif/ . 
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Table C-10. Confidence intervals for the percentage of schools in Common Core states reporting 
major challenges when planning or implementing new or revised state standards and 
aligned assessments: 2011-12 



School reported challenge 


School rated the 


as a major challenge 1 


challenge 


School challenge 

% 

Low Cl 

High Cl 

% 

Low Cl 

High Cl 

Insufficient funding to support instructional specialists 
or coaches to help teachers implement new standards 

43 

38.8 

47.7 

84 

81.2 

86.7 

Insufficient funding to purchase new instructional 
materials aligned with new standards 

42 

38.2 

46.5 

85 

83.0 

87.9 

Current assessments are not aligned with the new 
standards 

30 

26.3 

34.5 

86 

83.6 

89.1 

Inadequate quality or availability of state-developed 
instructional materials aligned with standards 

28 

24.5 

32.3 

86 

83.5 

88.6 

Lack of school staff or expertise to develop new 
curricula guides and instructional materials aligned 
with new standards 

25 

20.9 

29.9 

87 

84.0 

89.0 

Lack of school staff or expertise to provide guidance 
about or train educators on using new standards for 
their instruction 

20 

16.6 

23.0 

87 

85.0 

90.0 

Standardized assessments not available for enough 
subjects or grades 

16 

12.8 

18.8 

90 

87.5 

91.5 

Lack of clear district guidance or support on 
expectations concerning when and how standards 
should be implemented 

12 

9.3 

14.4 

86 

81.3 

90.9 

Concerns or opposition from school staff about 
additional assessments 

11 

7.3 

15.2 

93 

90.9 

94.6 

Lack of clear district guidance or support on specific 
content of new standards 

9 

7.3 

11.7 

86 

81.2 

91.0 

Concerns or opposition from school staff or staff 
unions about new standards 

8 

6.5 

10.1 

85 

82.3 

87.4 

Lack of school staff or expertise to train educators on 
how to administer assessments 

6 

4.4 

8.2 

93 

91.7 

95.2 

Concerns or opposition from parents or other 
community groups to additional assessments 

4 

2.4 

5.1 

90 

88.0 

92.1 

Concerns or opposition from parents or other 
community groups focused on new standards 

4 

1.9 

5.5 

84 

81.8 

87.1 


1 The challenge could be rated as not a challenge, as a minor challenge, or as a major challenge. 

2 Schools rating the challenge excludes schools that identified the challenge as not applicable, schools that did not respond to 
the question, and schools in states that did not adopt the CCSS in both reading/English language arts and mathematics. 

Notes: The percentages in the table are cross-sectional estimates for the population of schools in Common Core states that 
were operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. 
Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table C-ll. Results of significance tests comparing percentages of schools in Common Core states 
that reported major challenges when planning or implementing new or revised state 
standards and aligned assessments: 2011-12 

Ref. # Challenge Percent 1 

1 Insufficient funding to support instructional specialists or coaches to help teachers implement ^ 

new standards 

2 Insufficient funding to purchase new instructional materials aligned with new standards 42 

3 Current assessments are not aligned with the new standards 30 

4 Inadequate quality or availability of state-developed instructional materials aligned with 
standards 

5 Lack of school staff or expertise to develop new curricula guides and instructional materials ^ 

aligned with new standards 

6 Lack of school staff or expertise to provide guidance about or train educators on using new 
standards for their instruction 

7 Standardized assessments not available for enough subjects or grades 16 

8 Lack of clear district guidance or support on expectations concerning when and how 
standards should be implemented 

9 Concerns or opposition from school staff about additional assessments 11 

10 Lack of clear district guidance or support on specific content of new standards 9 

11 Concerns or opposition from school staff or staff unions about new standards 8 

12 Lack of school staff or expertise to train educators on how to administer assessments 6 

13 Concerns or opposition from parents or other community groups to additional assessments 4 

14 Concerns or opposition from parents or other community groups to new standards 4 

1 These are the percentage of schools in CCSS states that rated the challenge as major. They are cross-sectional estimates for 
the population of schools operating in the 2010-11 school year. See appendix A for more information about the 
generalizability of the estimates. 

(continued) 
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Table C-ll. Results of significance tests comparing percentages of schools in Common Core states 
that reported major challenges when planning or implementing new or revised state 
standards and aligned assessments: 2011-12 (cont'd) 


Ref. # 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

1 



* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

2 



* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

3 





* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

4 






* 

* 

* 

* 

* 

* 

* 

* 

* 

5 






* 

* 

* 

* 

* 

* 

* 

* 

* 

6 








* 

* 

* 

* 

* 

* 

* 

7 








* 


* 

* 

* 

* 

* 

8 










* 

* 

* 

* 

* 

9 












* 

* 

* 

10 












* 

* 

* 

11 













* 

* 

12 













* 


13 















14 
















^Percentages are significantly different (p < 0.05). 

Table reads: As denoted by the asterisk in row 1, column 2, a significantly larger percentage of schools rated insufficient funding 
to support instructional specialists or coaches to help teachers implement new standards as a major challenge (43 percent) 
than rated insufficient funding to purchase new instructional materials aligned with new standards as a major challenge (42 
percent). Use the reference number (Ref. #) to identify the challenge. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table D-l. Confidence intervals for percentage of districts that supported reforms related to 
educators' use of student data: 2009-10, 2010-11, and 2011-12 




2009-10 



2010-11 



2011-12 




Low 

High 


Low 

High 


Low 

High 

Indicator description 

% 

Cl 

Cl 

% 

Cl 

Cl 

% 

Cl 

Cl 

District Provided Educators with Access to 
Assessment Data 

91 

88.3 

94.7 

< 

00 

00 

83.1 

92.4 

89 

85.5 

92.0 

District Provided Educators with 
Professional Development on the Use of 
Assessment Data for Instructional Planning 

83 

76.2 

90.4 

84 

80.1 

88.4 

80 

74.4 

84.7 

District Used Longitudinal Data to Track 










Student Achievement Gains for Individual 
Teachers 

60 

54.7 

66.0 

56 

49.1 

62.8 

66 

59.2 

72.4 


Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < 0.05). 

§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < 0.05). 

Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. For 2010-11 and 2011- 
12, districts were asked whether they used or included the strategy in all schools, some schools, or if they were not using the 
strategy whether they were actively planning its use or had no plans to use or include the strategy. For the 2009-10 school year, 
the response options included only Yes and No. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 District Surveys. 
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Table D-2. Results of significance tests comparing percentages of districts that supported reforms 
related to educators' use of student data: 2011-12 


Ref# 

Indicator description 

Percent 
of districts 1 

1 

District Provided Educators with Access to Assessment Data 

89 

2 

District Provided Educators with Professional Development on the Use of Assessment Data 
for Instructional Planning 

80 

3 

District Used Longitudinal Data to Track Student Achievement Gains for Individual 
Teachers 

66 


1 These percentages show the percentage of districts that satisfied the requirements of the indicator. They are cross-sectional 
estimates for the population of districts operating in the 2010-11 school year. See appendix A for more information about the 
generalizability of these estimates. 


Ref# 

1 

2 

3 

1 


* 

* 

2 



* 

3 





*Percentages are significantly different (p < 0.05). 

Table reads: As denoted by the asterisk in row 1, column 2, a significantly larger percentage of districts provided educators with 
access to assessment data (89 percent) than provided educators with professional development on the use of assessment data 
for instructional planning (80 percent). Use the reference number (Ref #) to identify the indicator. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table D-3. Confidence intervals for percentage of high-poverty and other districts that supported 
reforms related to educators' use of student data: 2011-12 



High Poverty 



Other 


Indicator description 

% 

Low Cl 

High Cl 

% 

Low Cl 

High Cl 

District Provided Educators with Access to 
Assessment Data 

88 

82.9 

93.0 

89 

85.3 

92.9 

District Provided Educators with Professional 
Development on the Use of Assessment Data for 
Instructional Planning 

80 

72.9 

87.8 

79 

72.8 

85.8 

District Used Longitudinal Data to Track Student 
Achievement Gains for Individual Teachers 

68 

57.9 

77.9 

65 

56.6 

73.5 


Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this table, the 
denominator for the high poverty percentages is the estimated number of high-poverty districts (had a child poverty rate above 
21.66 percent) that have sufficient data. The denominator for the other percentages is the estimated number of districts with 
sufficient data that had that had a child poverty rate at or below 21.66 percent. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. U.S. Census Bureau. Small Area Income and Poverty 
Estimates (SAIPE) program, district data for 2008 (USSD08.xls) for poverty data. Retrieved November 11, 2010, from 
http://www.census.gov/did/www/saipe/data/schools/index.html . 
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Table D-4. Confidence intervals for percentage of large districts and districts of other sizes that 
supported reforms related to educators' use of student data: 2011-12 




Large 



Medium 



Small 


Indicator description 

% 

Low Cl 

High Cl 

% 

Low Cl 

High Cl 

% 

Low Cl 

High Cl 

District Provided Educators with 
Access to Assessment Data 
District Provided Educators with 

99 

98.8 

98.8 

93* 

90.0 

96.4 

89* 

85.1 

91.9 

Professional Development on the 
Use of Assessment Data for 
Instructional Planning 

94 

94.0 

94.0 

90* 

85.8 

93.3 

79* 

73.7 

84.5 

District Used Longitudinal Data to 
Track Student Achievement Gains 
for Individual Teachers 

73 

72.8 

72.8 

67* 

62.7 

70.4 

66* 

58.8 

72.6 


* Percentage is significantly different from percentage for large districts (p<0.05). 

Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this table, the 
denominator for a size group is the estimated number of districts of a given size with sufficient data. District size is based on the 
student enrollment in schools in the district. Large districts have at least 50,000 students. Medium districts have less than 
50,000 but at least 15,000 students. Small districts have fewer than 15,000 students. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. National Center for Education Statistics, Common Core of 
Data. Local Education Agency Universe Survey: School Year 2008-09 (ag081a.sas7bdat) for enrollment size. Retrieved August 
24, 2010, from http://nces.ed.gov/ccd/pubagency.asp . 
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Table D-5. Confidence intervals for percentage of districts that reported major challenges using 
student assessment data: 2011-12 


District reported challenge as a 

major challenge 1 Districts rated the challenge 2 


District challenge 

Percent 

Low Cl 

High Cl 

Percent 

Low Cl 

High Cl 

Delays in transmission of assessment results 
to schools or teachers 

35 

28.1 

41.1 

94 

87.6 

99.7 

Insufficient funding to: 







Support data systems that store and 
provide access to assessment 
information 

Train educators in how to administer 

33 

27.4 

39.2 

97 

93.9 

99.1 

and use assessments 

26 

20.0 

32.1 

96 

92.5 

99.1 

Lack of district staff capacity or expertise to: 







Provide guidance about or train 
educators on how to use assessments 
to improve instruction 

19 

11.2 

26.0 

97 

94.9 

99.7 

Maintain and facilitate educators' 
access to assessment data systems 

18 

13.6 

21.5 

97 

94.1 

99.1 

Lack of clear SEA guidance or support on 
using state assessment data systems 

14 

9.9 

18.1 

87 

80.3 

92.7 

Restrictions in rules and regulations relating 
to what can be included in state or district 

14 

8.8 

18.6 

85 

78.4 

91.5 


data systems and how to access them 


1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Districts rating the challenge excludes districts that identified the challenge as not applicable and districts that did not 
respond to the question. 

Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table D-6. Results of significance tests comparing percentages of districts that reported major 
challenges using student assessment data: 2011-12 


Ref. # Challenge description Percent 1 

1 Delays in transmission of assessment results to schools or teachers 35 

Insufficient funding to: 

2 Support data systems that store and provide access to assessment information 33 

3 Train educators in how to administer and use assessments 26 

Lack of district staff capacity or expertise to: 


4 Provide guidance about or train educators on how to use assessments to improve 
instruction 

5 Maintain and facilitate educators' access to assessment data systems 18 

6 Lack of clear SEA guidance or support on using state assessment data systems 14 

7 Restrictions in rules and regulations relating to what can be included in state or district data , 

systems and how to access them 


1 These are the percentage of districts that rated the challenge as major. They are cross-sectional estimates for the population 
of districts operating in the 2010-11 school year. See appendix A for more information about the generalizability of the 
estimates 


Ref. # 

1 

2 

3 

4 

5 

6 

7 

1 



* 

* 

* 

* 

* 

2 



* 

* 

* 

* 

* 

3 





* 

* 

* 

4 








5 








6 








7 









*Percentages are significantly different (p < 0.05). 

Table reads: As denoted by no asterisk in row 1, column 2, the percentage of districts that rated delays in transmission of 
assessment results to schools or teachers as a major challenge (35 percent) is not significantly different from the percentage 
that rated insufficient funding to support data systems that store and provide access to assessment information (33 percent). 
Use the reference number (Ref. #) to identify the challenge. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table D-7. Confidence intervals for percentage of schools that used student data to support 
instruction: 2009-10, 2010-11, and 2011-12 




2009-10 



2010-11 



2011-12 




Low 

High 


Low 

High 


Low 

High 

Indicator description 

% 

Cl 

Cl 

% 

Cl 

Cl 

% 

Cl 

Cl 

School Used Student Assessment Data to 
Identify Students for Additional Support 

93 

90.7 

95.2 

< 

00 

cr> 

95.0 

100.0 

98+ 

96.8 

98.7 

Teachers Had On-Line Access to Student 
Assessment Results 

85 

82.8 

87.7 

A 

90 

88.2 

92.6 

92+ 

89.8 

94.4 

School Used Student Assessment Data to 
Tailor Instruction 

85 

82.3 

87.3 

A 

94 

90.9 

96.5 

95+ 

93.6 

96.1 

School Used Longitudinal Data to Track 
Student Achievement Gains for Individual 
Teachers 

60 

57.4 

63.1 

68 A 

65.7 

71.3 

71+ 

68.0 

74.0 


Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < 0.05). 

^Percentage for 2011-12 is significantly different from percentage for 2009-10 (p < 0.05). 

Notes: The percentages in the table are cross-sectional estimates for the population of schools that were operating in the 2010- 
11 school year. See appendix A for more information about the generalizability of the estimates. Detailed tables in appendix D 
provide confidence intervals for each percentage. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. 
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Table D-8. Results of significance tests comparing percentages of schools that used student data to 
support instruction: 2011-12 


Percent 

Ref# Indicator description of schools 1 

1 School Used Student Assessment Data to Identify Students for Additional Support 98 

2 Teachers Had On-Line Access to Student Assessment Results 92 

3 School Used Student Assessment Data to Tailor Instruction 95 

4 School Used Longitudinal Data to Track Student Achievement Gains for Individual Teachers 71 


1 These percentages show the percentage of schools that satisfied the requirements of the indicator. They are cross-sectional 
estimates for the population of schools operating in the 2010-11 school year. See appendix A for more information about the 
generalizability of the estimates. 



*Percentages are significantly different (p < .05). 

Table reads: As denoted by the asterisk in row 1, column 2, a significantly larger percentage of schools used assessment data to 
identify students for additional support (98 percent) than reported that their teachers had on-line access to student assessment 
results (92 percent). Use the reference number (Ref #) to identify the indicator. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table D-9. Confidence intervals for percentage of low-performing and other schools that used 
student data to support instruction: 2011-12 


Low-performing 


schools 

Other schools 

Low High 

Low High 


Indicator description 

% 

Cl 

Cl 

% 

Cl 

Cl 

School Used Student Assessment Data to Identify Students for 
Additional Support 

99 

98.1 

100.0 

* 

97 

96.3 

98.6 

Teachers Had On-Line Access to Student Assessment Results 

94 

91.4 

97.3 

92 

88.9 

94.3 

School Used Student Assessment Data to Tailor Instruction 

97 

94.8 

99.1 

94 

92.9 

95.8 

School Used Longitudinal Data to Track Student Achievement 
Gains for Individual Teachers 

77 

71.6 

81.8 

70* 

66.2 

73.5 


* Percentage is significantly different from percentage for low-performing schools (p<0.05). 

Notes: The percentages in the table are cross-sectional estimates for the population of schools that were operating in the 2010- 
11 school year. See appendix A for more information about the generalizability of the estimates. In this table, the denominator 
is the estimated number of schools in the performance status category (low-performing, other) with sufficient data. Low- 
performing schools include schools that were in improvement, corrective action, or restructuring, were identified as among the 
lowest achieving schools, or that have had a graduation rate below 60 percent over a number of years. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. Approved state applications for School Improvement grants 
for school performance status. Retrieved December 2010 from http://www2.ed.gov/programs/sif/ . 
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Table D-10. Confidence intervals for percentage of schools that reported major challenges using data 
systems for storing, reporting, and using assessment results: 2011-12 



School reported challenge as 





a major challenge 1 

School rated the challenge 

School challenge 

Percent 

Low Cl 

High Cl 

Percent 

Low Cl 

High Cl 

Delays in transmission of assessment results to 
school or teachers 

21 

17.8 

23.8 

93 

91.1 

94.2 

Insufficient funding to purchase or sustain data 
systems that store and provide access to 
assessment information 

18 

15.6 

21.3 

90 

87.7 

91.3 

Lack of school staff or expertise to: 







Train educators on how to use 
assessments to improve instruction 

11 

7.7 

15.2 

94 

92.4 

95.4 

Maintain and facilitate educators' 
access to assessment data systems 

10 

8.0 

12.0 

94 

92.2 

95.3 

Lack of clear district guidance or support on 
using state and district assessment data systems 

8 

6.1 

9.3 

90 

86.1 

94.2 


1 The challenge could be rated as not a challenge, as a minor challenge, or as a major challenge. 

2 Schools rating the challenge excludes schools that identified the challenge as not applicable and schools that did not respond 
to the question. 

Notes: The percentages in the table are cross-sectional estimates for the population of schools that were operating in the 2010- 
11 school year. See appendix A for more information about the generalizability of the estimates. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table D-ll. Results of significance tests comparing percentages of schools that reported major 
challenges using data systems for storing, reporting, and using assessment results: 
2011-12 


Ref. # 

Challenge 

Percent 1 

ql9i 

Delays in transmission of assessment results to school or teachers 

21 

ql9a 

Insufficient funding to purchase or sustain data systems that store and provide access to 
assessment information 

18 


Lack of school staff or expertise to: 


ql9c 

Train educators on how to use assessments to improve instruction 

11 

ql9d 

Maintain and facilitate educators' access to assessment data systems 

10 

ql9e 

Lack of clear district guidance or support on using state and district assessment data 
systems 

8 


1 These are the percentage of schools that rated the challenge as major. They are cross-sectional estimates for the population 
of schools operating in the 2010-11 school year. See appendix A for more information about the generalizability of the 
estimates. 


Ref. # 

ql9i 

ql9a 

ql9c 

ql9d 

ql9e 

ql9i 



* 

* 

* 

ql9a 



* 

* 

* 

ql9c 






ql9d 





* 

ql9e 







^Percentages are significantly different (p < 0.05). 

Table reads: As denoted by no asterisk in row 1, column 2, the percentage of schools that rated delays in transmission of 
assessment results to schools or teachers as a major challenge (21 percent) was not significantly different from the percentage 
that rated insufficient funding to purchase or sustain data systems that store and provide access to assessment information as 
major (18 percent). Use the reference number (Ref. #) to identify the challenge. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table E-l. Number and percent of SEAs that simplified/shortened educator licensure process or 
authorized non-university preparation programs, by SEA reform strategy component: 
2009-10 to 2011-12 



2009-10 

2010-11 

2011-12 

Reform strategy 

Number 

Percent 

Number 

Percent 

Number 

Percent 


Reported pursuing the following strategy/strategies related to educator licensure and certification: 


Simplifying or shortening the process of obtaining full licensure and/or certification for: 


State-university-based teacher 
preparation programs 

16 

3 

18 

35 

19 

37 

(additional states that reported in 
2009-10 only) 



(1) 

(2) 

(1) 

(2) 

(additional states that reported in 
2010-11 not 2011-12) 





(7) 

(14) 

Alternative pathway teacher 
preparation programs 

22 

43 

26 

51 

24 

47 

(additional states that reported in 
2009-10 only) 



(1) 

(2) 

(1) 

(2) 

(additional states that reported in 
2010-11 not 2011-12) 





(8) 

(16) 

Authorized independent providers to 
provide teacher training 

16 

31 

17 

33 

22 

43 

(additional states that reported in 
2009-10 only) 



(1) 

(2) 

(1) 

(2) 

(additional states that reported in 
2010-11 not 2011-12) 





(4) 

(8) 

Met Indicator (educator licensure and 
certification reforms) 1 

33 

65 

35 

69 

39 

76 


1 An SEA met the indicator if it reported pursuing one or more of the three strategies related to teacher licensure and 
certification. 

Notes: Respondents include 50 states and DC. 

Numbers in parentheses are SEAs that reported the strategy prior to 2011-12 but not in 2011-12. These counts are included for 
strategies that may not be expected to occur annually, where 2009-10 or 2010-11 activity could reflect ongoing reform efforts. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 State Education Agencies Survey. 
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Table E-2. Number and percentage of SEAs that supported use of multi-level ratings, multiple 

observations, and student achievement gains for teacher evaluation, by SEA method of 
support and evaluation component: 2011-12 



SEAs 

Indicator description 

Number 

Percent 

Method of SEA support: 



Administering a state-developed teacher evaluation system in which LEA 
participation is: 



Required 

13 

25 

Optional 

16 

26 

Setting standards and guidelines for LEA-designed systems that are: 



Required 

24 

47 

Optional 

10 

20 

Providing guidance/technical assistance to LEAs on system design and 
implementation 

31 

61 

Requiring LEAs to submit teacher evaluation design and implementation plans for 
SEA approval 

12 

24 

Requiring LEAs to report on teacher evaluation system operations and effectiveness 

14 

27 

At least one of the above 

42 

82 

Components of SEA system: 



Use rating scales or rubrics that define three or more performance levels 

28 

55 

Include at least two annual observations of classroom instruction accompanied by 
written feedback 

19 

37 

Use student achievement gains in NCLB-tested subjects and grades to determine 
teacher performance ratings 

20 

39 

MET INDICATOR (at least one role and all components) 

14 

27 


Note: Respondents include 50 states and DC. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 State Education Agencies Survey. 
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Table E-3. Confidence intervals for percentage of districts that implemented reforms related to 
educator workforce development: 2009-10, 2010-11, and 2011-12 




2009-10 



2010-11 



2011-12 




Low 

High 


Low 

High 


Low 

High 

Indicator description 

% 

Cl 

Cl 

% 

Cl 

Cl 

% 

Cl 

Cl 

District Provided School Leaders with 
Professional Development or Flexibility 
to Hire Effective Teachers 

69 

62.5 

75.9 

64 

58.5 

70.2 

63+ 

56.1 

69.8 

District Used Student Achievement Gains 
for Teacher Tenure, Dismissal, or 
Assignment Decisions 

38 

33.1 

43.6 

34 

27.2 

40.7 

24t§ 

19.7 

29.0 


District Operated a Principal Evaluation 
System that Included Student 
Achievement Gains 

36 

29.7 

42.2 

30 

25.5 

34.2 

30 

24.8 

35.9 

District Operated a Teacher Evaluation 
System that Included Multi-level Rubrics, 
Multiple Observations, and Student 
Achievement Gains 

15 

10.9 

18.9 

20 

12.8 

27.1 

17 

13.2 

20.0 

District Differentiated Teacher 
Compensation Based On Student 
Achievement Gains 

12 

9.9 

14.0 

16 

9.1 

22.5 

8+ § 

5.5 

11.1 



^Percentage for 2011-12 is significantly different from percentage for 2009-10 (p < 0.05). 

§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < 0.05). 

Notes: The percentages in this table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. 

For 2010-11 and 2011-12, districts were asked whether they used or included the strategy in all schools, some schools, or if 
they were not using the strategy whether they were actively planning its use or had no plans to use or include the strategy. For 
the 2009-10 school year, the response options included only Yes and No. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 District Surveys. 
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Table E-4. Results of significance tests comparing percentages of districts that implemented reforms 
related to educator workforce development: 2011-12 


Ref. # 

Indicator description 

Percent 
of districts 1 

1 

District provided school leaders with professional development or flexibility to hire 
effective teachers 

63 

2 

District Used Student Achievement Gains for Teacher Tenure, Dismissal, or 
Assignment Decisions 

24 

3 

District Operated a Principal Evaluation System that Included Student Achievement 
Gains 

30 

4 

District Operated a Teacher Evaluation System that Included Multi-level Rubrics, 
Multiple Observations, and Student Achievement Gains 

17 

5 

District Differentiated Teacher Compensation Based On Student Achievement Gains 

8 


1 These percentages show the percentage of districts that satisfied the requirements of the indicator. They are cross-sectional 
estimates for the population of districts operating in the 2010-11 school year. See appendix A for more information about the 
generalizability of these estimates. 


Ref. # 

1 

2 

3 

4 

5 

1 


* 

* 

* 

* 

2 



* 

* 

* 

3 




* 

* 

4 





* 

5 







^Percentages are significantly different (p <.0.05). 

Table reads: As denoted by the asterisk in row 1, column 2, a significantly larger percentage of districts provided school leaders 
with professional development or flexibility to hire effective teachers (63 percent) than used student achievement gains for 
teacher tenure, dismissal, or assignment decisions (24 percent). Use the reference number (Ref #) to identify the indicator. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table E-5. Percentage of districts that provided school leaders with professional development or 

flexibility to hire effective teachers, by reform strategy component: 2009-10, 2010-11 and 

2011-12 


Reform strategy 

Percent 

2009-10 

2010-11 

2011-12 

Provide PD for principals on identifying, recruiting and hiring 
effective teachers 

55 

48 

50 

Provide school leaders with authority to hire more qualified transfer 
candidates without regard to seniority 

45 

48 

46 

MET INDICATOR (at least one of the above) 

69 

64 

63 


Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. The denominator for the reform strategy percentages is the number of districts that responded to the 
question. The denominator for the indicator is the number of districts with sufficient data (i.e., answered enough questions to 
calculate the indicator). 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 District Survey. 
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Table E-6. Percentage of districts that operated a teacher evaluation system that included multi- 
level rubrics, multiple observations, and student achievement gains: 2011-12 


Teacher evaluation system components: Percent 

Uses a rating scale or rubric that defines three or more performance levels to evaluate classroom ^ 

instruction or practice 

Includes at least two yearly observations of classroom instruction with written feedback 78 

Includes student achievement gains in NCLB grades/subjects in determining individual teacher 
performance ratings 

MET INDICATOR (teacher evaluation system included all of the components listed ) 17 

Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table E-7. Confidence intervals for percentage of high-poverty and other districts that implemented 
reforms related to educator workforce development: 2011-12 




High Poverty 



Other 


Indicator description 

% 

Low Cl 

High Cl 

% 

Low Cl 

High Cl 

District Provided School Leaders with 
Professional Development or Flexibility to Hire 
Effective Teachers 

57 

49.9 

64.4 

65 

56.4 

73.7 

District Used Student Achievement Gains for 
Teacher Tenure, Dismissal, or Assignment 
Decisions 

29 

21.2 

37.5 

22 

16.8 

28.2 

District Operated a Principal Evaluation System 
that Included Student Achievement Gains 

37 

30.3 

44.0 

28 

20.6 

35.0 

District Operated a Teacher Evaluation System 
that Included Multi-level Rubrics, Multiple 
Observations, and Student Achievement Gains 

21 

11.1 

30.9 

15 

12.2 

17.7 

District Differentiated Teacher Compensation 
Based On Student Achievement Gains 

16 

6.9 

24.8 

5* 

3.4 

7.6 


^Percentage is significantly different from percentage for high-poverty districts (p<0.05). 

Notes: The percentages in this table are cross-sectional estimates for the population of districts operating in the 2010-11 school 
year. The denominator for the high poverty percentages is the estimated number of high-poverty districts (had a child poverty 
rate above 21.66 percent) that have sufficient data. The denominator for the other percentages is the estimated number of 
districts with sufficient data that had a child poverty rate at or below 21.66 percent. See appendix A for more information about 
the generalizability of the estimates. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. U.S. Census Bureau. Small Area Income and Poverty 
Estimates (SAIPE) program, district data for 2008 (USSD08.xls) for poverty data. Retrieved November 11, 2010, from 
http://www.census.eov/did/www/saipe/data/schools/index.html . 
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Table E-8. Confidence intervals for percentage of large districts and districts of other sizes that 
implemented reforms related to educator workforce development: 2011-12 




Large 



Medium 



Small 


Indicator description 

% 

Low Cl 

High Cl 

% 

Low Cl 

High Cl 

% 

Low Cl 

High Cl 

District Provided School Leaders 
with Professional Development or 
Flexibility to Hire Effective 
Teachers 

74 

73.7 

73.7 

71 

66.7 

75.1 

63* 

55.3 

69.7 

District Used Student 
Achievement Gains for Teacher 
Tenure, Dismissal, or Assignment 
Decisions 

34 

34.1 

34.1 

26* 

22.0 

30.3 

24* 

19.3 

29.1 

District Operated a Principal 
Evaluation System that Included 
Student Achievement Gains 

59 

59.2 

59.2 

50* 

46.2 

54.6 

29* 

23.4 

35.2 

District Operated a Teacher 
Evaluation System that Included 
Multi-level Rubrics, Multiple 
Observations, and Student 
Achievement Gains 

37 

37.4 

37.4 

22* 

17.9 

26.6 

16* 

12.7 

19.8 

District Differentiated Teacher 
Compensation Based On Student 
Achievement Gains 

38 

37.9 

37.9 

12* 

9.5 

14.9 

8* 

5.0 

10.9 


^Percentage is significantly different from percentage for large districts (p<0.05). 

Notes: The percentages in this table are cross-sectional estimates for the population of districts operating in the 2010-11 school 
year. The denominator for a size group is the estimated number of districts of a given size with sufficient data. District size is 
based on the student enrollment in schools in the district. Large districts have at least 50,000 students. Medium districts have 
less than 50,000 but at least 15,000 students. Small districts have fewer than 15,000 students. See appendix A for more 
information about the generalizability of the estimates. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. National Center for Education Statistics, Common Core of 
Data. Local Education Agency Universe Survey: School Year 2008-09 (ag081a.sas7bdat) for enrollment size. Retrieved August 
24, 2010, from http://nces.ed.gov/ccd/pubaeencv.asp . 
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Table E-9. Confidence intervals for percentage of districts that reported major challenges when 
implementing educator evaluation and compensation systems: 2011-12 


District reported challenge Districts rated the 
as a major challenge 1 challenge 2 


District challenge 

% 

Low Cl 

High Cl 

% 

Low Cl 

High Cl 

Insufficient funding to provide differential 
compensation for teachers in high-need areas (e.g., 
low-performing schools, STEM subjects) 

84 

79.6 

88.9 

53 

46.5 

60.5 

Insufficient funding to provide performance-based 
compensation to all eligible teachers 

83 

78.1 

88.5 

57 

49.0 

65.1 

Concerns or opposition from school staff or staff 
unions about performance-based compensation 

59 

48.5 

69.5 

73 

66.3 

79.6 

Difficulty in measuring student growth for teachers of 
non-tested subjects 

59 

51.3 

66.4 

88 

84.9 

92.0 

Restrictions in rules and regulations on how educators 
can be compensated 

56 

48.6 

62.8 

74 

67.0 

81.2 

Concerns or opposition from school staff or staff 
unions about evaluating educators based, at least in 
part, on student achievement 

50 

40.4 

59.4 

80 

73.6 

85.5 

Lack of clear SEA guidance or support on educator 
compensation or evaluation system 

36 

27.2 

45.7 

78 

71.5 

84.0 

Restrictions in rules and regulations on how educators 
can be evaluated 

36 

26.9 

45.1 

84 

77.9 

89.9 

Lack of district staff capacity or expertise to conduct 
comprehensive educator performance evaluations 

26 

16.8 

35.0 

85 

79.6 

89.8 

Lack of district staff capacity or expertise to develop 
reliable approaches for rating educator performance- 
based, in part, on student achievement 

22 

15.6 

28.2 

78 

72.2 

82.9 

Current data systems make linking student test data to 
individual teachers difficult 

19 

13.9 

23.3 

87 

82.9 

92.1 

Lack of district staff capacity or expertise to identify 
professional development needs of teachers based on 

10 

6.1 

14.9 

87 

82.7 

91.3 


performance evaluations 


1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. 

2 Districts rating the challenge excludes districts that identified the challenge as not applicable and districts that did not 
respond to the question. 

Notes: The percentages in this table are cross-sectional estimates for the population of districts operating in the 2010-11 school 
year. See appendix A for more information about the generalizability of the estimates. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 


E-9 


Table E-10. Results of significance tests comparing percentages of districts that reported major 

challenges when implementing educator evaluation and compensation systems: 2011-12 

Ref. # Challenge description Percent 1 

1 Insufficient funding to provide differential compensation for teachers in high-need areas (e.g., 
low-performing schools, STEM subjects) 

2 Insufficient funding to provide performance-based compensation to all eligible teachers 83 

3 Concerns or opposition from school staff or staff unions about performance-based 
compensation 

4 Difficulty in measuring student growth for teachers of non-tested subjects 59 

5 Restrictions in rules and regulations on how educators can be compensated 56 

6 Concerns or opposition from school staff or staff unions about evaluating educators based, at ^ 

least in part, on student achievement 

7 Lack of clear SEA guidance or support on educator compensation or evaluation system 36 

8 Restrictions in rules and regulations on how educators can be evaluated 36 

9 Lack of district staff capacity or expertise to conduct comprehensive educator performance ^ 

evaluations 

10 Lack of district staff capacity or expertise to develop reliable approaches for rating educator ^ 

performance-based, in part, on student achievement 

11 Current data systems make linking student test data to individual teachers difficult 19 

12 Lack of district staff capacity or expertise to identify professional development needs of 
teachers based on performance evaluations 

1 These are the percentage of districts that rated the challenge as major. They are cross-sectional estimates for the population 

of districts operating in the 2010-11 school year. See appendix A for more information about the generalizability of the 

estimates. 

(continued) 
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Table E-10. Results of significance tests comparing percentages of districts that reported major 

challenges when implementing educator evaluation and compensation systems: 2011-12 
(cont'd) 


Ref.# 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

1 



* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

2 



* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

3 






* 

* 

* 

* 

* 

* 

* 

4 







* 

* 

* 

* 

* 

* 

5 







* 

* 

* 

* 

* 

* 

6 







* 

* 

* 

* 

* 

* 

7 









* 

* 

* 

* 

8 









* 

* 

* 

* 

9 












* 

10 












* 

11 












* 

12 














^Percentages are significantly different (p < 0.05). 

Table reads: As denoted by no asterisk in row 1, column 2, the percentage of districts that reported insufficient funding to 
provide differential compensation for teachers in high-need areas as a major challenge (84 percent) was not significantly 
different from the percentage reporting insufficient funding to provide performance-based compensation to all eligible 
teachers as a major challenge (83 percent). Use the reference number (Ref. #) to identify the challenge. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table E-ll. Confidence intervals for percentage of schools that used student achievement gains for 
educator evaluation, compensation, and personnel decisions: 2009-10, 2010-11, and 
2011-12 




2009-10 



2010-11 



2011-12 




Low 

High 


Low 

High 


Low 

High 

Indicator description 

% 

Cl 

Cl 

% 

Cl 

Cl 

% 

Cl 

Cl 

Principal Evaluation Practices Included 
Student Achievement Gains 

45 

41.6 

48.7 

A 

50 

46.5 

53.2 

49 

46.1 

52.1 

Teacher Tenure, Dismissal, or 
Reassignment Decisions Used Student 
Achievement Gains 

29 

26.3 

31.0 

A 

31 

28.7 

33.8 

34 + 

31.1 

36.2 

Teacher Evaluation Practices Included 
Multi-level Rubrics, Multiple 
Observations, and Student Achievement 
Gains 

18 

15.1 

21.4 

A 

22 

19.6 

24.6 

§ 

18 

16.1 

20.1 

Teacher Compensation Differentiated 
Based On Student Achievement Gains 

12 

9.7 

14.2 

12 

10.4 

14.5 

§ 

10 

8.3 

11.2 


Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < 0.05). 

f Percentage for 2011-12 is significantly different from percentage for 2009-10 (p < 0.05). 

5 Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < 0.05). 

Notes: The percentages in this table are cross-sectional estimates for the population of schools operating in the 2010-11 school 
year. See appendix A for more information about the generalizability of the estimates. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. 
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Table E-12. Results of significance tests comparing percentages of schools that used student 

achievement gains for educator evaluation, compensation, and personnel decisions: 

2011-12 


Ref.# 

Indicator description 

Percent 
of schools 1 

1 

Principal Evaluation Practices Included Student Achievement Gains 

49 

2 

Teacher Tenure, Dismissal, or Reassignment Decisions Used Student Achievement 
Gains 

34 

3 

Teacher Evaluation Practices Included Multi-level Rubrics, Multiple Observations, and 
Student Achievement Gains 

18 

4 

Teacher Compensation Differentiated Based On Student Achievement Gains 

10 

1 These percentages show the percentage of schools that satisfied the requirements of the indicator. They are cross-sectional 
estimates for the population of schools operating in the 2010-11 school year. See appendix A for more information about the 
generalizability of the estimates. 


Ref. # 1 

2 

3 

4 

1 

* 

* 

* 

2 


* 

* 

3 



* 


4 


^Percentages are significantly different (p < 0.05) 

Table reads: As denoted by the asterisk in row 1, column 2, a significantly larger percentage of schools' principal evaluation 
practices included student achievement gains (49 percent) than used student achievement gains for teacher tenure, dismissal, 
or assignment decisions (34 percent). Use the reference number (Ref#) to identify the indicator. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table E-13. Confidence intervals for percentage of low-performing and other schools that used 
student achievement gains for educator evaluation, compensation, and personnel 
decisions: 2011-12 



Low-performing 

schools 

Other schools 

Indicator description 

% 

Low 

Cl 

High 

Cl 

% 

Low 

Cl 

High 

Cl 

Principal Evaluation Practices Included Student Achievement 
Gains 

57 

51.3 

63.7 

47* 

44.3 

50.4 

Teacher Tenure, Dismissal, or Reassignment Decisions Used 
Student Achievement Gains 

35 

28.1 

42.7 

33 

30.5 

36.0 

Teacher Evaluation Practices Included Multi-level Rubrics, 
Multiple Observations, and Student Achievement Gains 

20 

14.3 

26.0 

18 

15.4 

20.0 

Teacher Compensation Differentiated Based On Student 
Achievement Gains 

14 

10.3 

18.6 

9 * 

7.2 

10.4 


* Percentage is significantly different from percentage for low-performing schools (p<0.05). 

Notes: The percentages in this table are cross-sectional estimates for the population of schools operating in the 2010-11 school 
year. The denominator is the estimated number of schools in the performance status category (low-performing, other) with 
sufficient data. Low-performing schools include schools that were (1) in improvement, corrective action, or restructuring, (2) 
identified as among the lowest achieving schools, or (3) have had a graduation rate below 60 percent over a number of years. 
Performance data obtained from approved state applications for School Improvement Grants. See appendix A for more 
information about the generalizability of the estimates. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. Approved state applications for School Improvement Grants 
for school performance status. Retrieved December 2010 from http://www2.ed.gov/programs/sif/ . 
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Table E-14. Confidence intervals for percentage of schools that reported major challenges when 
implementing educator evaluation and compensation systems: 2011-12 


School reported challenge as a 

major challenge 1 School rated the challenge 2 


School challenge 

Percent 

Low Cl 

High Cl 

Percent 

Low Cl 

High Cl 

Insufficient funding to provide performance- 
based compensation to all eligible teachers 

73 

69.2 

76.2 

49 

45.5 

52.8 

Insufficient funding to provide differential 
compensation for teachers in high-need 
areas (e.g., STEM subjects) 

70 

66.7 

73.3 

47 

43.8 

50.9 

Concerns or opposition from school staff or 
staff unions about performance-based 
compensation 

55 

50.6 

59.3 

64 

61.1 

67.1 

Difficulty in measuring student growth for 
teachers of non-tested subjects 

47 

43.8 

50.6 

81 

78.9 

84.1 

Restrictions in rules and regulations on how 
educators can be compensated 

47 

42.2 

51.0 

67 

64.2 

70.1 

Concerns or opposition from school staff or 
staff unions about evaluating educators 
based, at least in part, on student 
achievement 

41 

37.7 

45.0 

74 

71.4 

76.7 

Restrictions in rules and regulations on how 
educators can be evaluated 

23 

20.0 

26.6 

84 

81.4 

86.8 

Lack of school staff capacity or expertise to 
conduct comprehensive educator 
performance evaluations 

16 

12.4 

19.6 

84 

81.8 

86.5 

Lack of clear district guidance or support on 
educator compensation or evaluation system 

15 

12.3 

16.9 

71 

66.9 

74.7 

Limited access to technology needed in order 
to link student test data to individual 
teachers 

11 

8.7 

12.7 

83 

79.8 

86.8 

Lack of school staff capacity or expertise to 
Identify professional development needs of 

8 

5.6 

9.9 

86 

83.7 

88.2 


teachers based on performance evaluations 


^he challenge could be rated as not a challenge, as a minor challenge, or as a major challenge. 

2 Schools rating the challenge excludes schools that identified the challenge as not applicable and schools that did not respond 
to the question. 

Notes: The percentages in this table are cross-sectional estimates for the population of schools operating in the 2010-11 school 
year. See appendix A for more information about the generalizability of the estimates. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table E-15. Results of significance tests comparing percentages of schools that reported major 

challenges when implementing educator evaluation and compensation systems: 2011-12 

Ref. # Challenge description Percent 1 

1 Insufficient funding to provide performance-based compensation to all eligible teachers 73 

2 Insufficient funding to provide differential compensation for teachers in high-need areas 
(e.g., STEM subjects) 

3 Concerns or opposition from school staff or staff unions about performance-based 
compensation 

4 Difficulty in measuring student growth for teachers of non-tested subjects 47 

5 Restrictions in rules and regulations on how educators can be compensated 47 

6 Concerns or opposition from school staff or staff unions about evaluating educators based, 
at least in part, on student achievement 

7 Restrictions in rules and regulations on how educators can be evaluated 23 

8 Lack of school staff capacity or expertise to conduct comprehensive educator performance 
evaluations 

9 Lack of clear district guidance or support on educator compensation or evaluation system 15 

10 Limited access to technology needed in order to link student test data to individual teachers 11 

11 Lack of school staff capacity or expertise to identify professional development needs of 
teachers based on performance evaluations 

1 These are the percentage of schools that rated the challenge as major. They are cross-sectional estimates for the population 

of schools operating in the 2010-11 school year. See appendix A for more information about the generalizability of the 

estimates. 

(continued) 
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Table E-15. Results of significance tests comparing percentages of schools that reported major 

challenges when implementing educator evaluation and compensation systems: 2011-12 
(cont'd) 

Ref. #12 3 

4 

5 

6 

7 

8 

9 

10 

11 

1 * 

* 

* 

* 

* 

* 

* 

* 

* 

2 * 

* 

* 

* 

* 

* 

* 

* 

* 

3 

* 

* 

* 

* 

* 

* 

* 

* 

4 



* 

* 

* 

* 

* 

* 

5 



* 

* 

* 

* 

* 

* 

6 




* 

* 

* 

* 

* 

7 





* 

* 

* 

* 

8 







* 

* 

9 







* 

* 

10 








* 

11 










^Percentages are significantly different (p < 0.05). 

Table reads: As denoted by no asterisk in row 1, column 2, the percentage of schools that reported insufficient funding to 
provide performance-based compensation to all eligible teachers as a major challenge (73 percent) is not significantly different 
from the percentage reporting insufficient funding to provide differential compensation for teachers in high-need areas as a 
major challenge (70 percent). Use the reference number (Ref. #) to identify the challenge. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table F-l. Percentage of districts that replaced principal and teachers in low-performing schools, 

2011-12 


District policies or programs to support school restructuring or reorganization: Percent 

Replace a substantial proportion of the teachers in individual low-performing schools 6 

Replace principals in individual low-performing schools 16 

At least one of the above 17 

MET INDICATOR (district replaced teachers and principal) 5 


Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table F-2. Confidence intervals for percentage of districts with low-performing schools that 
implemented reforms to support improvement in low-performing schools: 2011-12 


2011-12 



Low 


Indicator description 

% 

Cl 

High Cl 

District Implemented Programs in Low-Performing Schools to Encourage Family and 
Community Involvement 

78 

63.7 

92.6 

District Provided Technical Assistance to Low-Performing Schools to Screen or Select 
School Improvement Experts or Models 1 

50 

38.0 

62.4 

District Extended School Day, Week, or Year in Low-Performing Schools 

40 

26.7 

53.4 

District Required Low-Performing Schools to Partner with Organizations that 
Specialize in Instructional Improvement 

39 

29.4 

49.6 

District Provided School Leaders in Low-Performing Schools with Staffing or Budgeting 
Flexibility to Implement School Reform 

23 

12.6 

33.2 

District Provided Compensation Incentives to Improve Staffing at Low-Performing 
Schools 

16 

8.4 

22.8 

District Replaced Principal and Teachers in Low-Performing Schools 1 

5 

3.5 

6.4 

District Contracted with External Organization to Operate Low-Performing Schools 

3 

2.2 

4.4 

District Targeted Low-Performing Schools for Closure 1 

2 

1.1 

3.5 


1 Districts responding to the evaluation's survey may have interpreted the improvement strategies included in this indicator as 
referring to actions that occur on a less than annual basis. As a consequence, these indicators may underestimate the 
percentage of districts supporting improvement using these strategies. 

Notes: The percentages in the figure are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year and have low-performing schools. See appendix A for more information about the generalizability of the 
estimates. In this figure, the denominator is the estimated number of districts with sufficient data that reported in the 
evaluation's 2012 district survey they had low-performing schools in that year. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table F-3. Results of significance tests comparing percentages of districts with low-performing 

schools that implemented reforms to support improvement in low-performing schools: 

2011-12 


Ref. # 

Indicator description 

Percent 
of disricts 1 

1 

District Implemented Programs in Low-Performing Schools to Encourage Family and 
Community Involvement 

78 

2 

District Provided Technical Assistance to Low-Performing Schools to Screen or Select 
School Improvement Experts or Models 

50 

3 

District Extended School Day, Week, or Year in Low-Performing Schools 

40 

4 

District Required Low-Performing Schools to Partner with Organizations that Specialize in 
Instructional Improvement 

39 

5 

District Provided School Leaders in Low-Performing Schools with Staffing or Budgeting 
Flexibility to Implement School Reform 

23 

6 

District Provided Compensation Incentives to Improve Staffing at Low-Performing Schools 

16 

7 

District Replaced Principal and Teachers in Low-Performing Schools 

5 

8 

District Contracted with External Organization to Operate Low-Performing Schools 

3 

9 

District Targeted Low-Performing Schools for Closure 

2 


1 These are the percentage of districts with low performing schools that rated the challenge as major. They are cross-sectional 
estimates for the population of districts operating in the 2010-11 school year. See appendix A for more information about the 
generalizability of the estimates. 



*Percentages are significantly different (p < 0.05). 

Table reads: As denoted by the asterisk in row 1, column 2, a significantly larger percentage of districts implemented programs 
in low-performing schools to encourage family and community involvement (78 percent) than provided technical assistance to 
low-performing schools to screen or select school improvement experts or models (50 percent). Use the reference number 
(Ref. #) to identify the challenge. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table F-4. Confidence intervals for percentage of districts with low-performing schools that 

reported major challenges when supporting school restructuring and improvement: 

2011-12 



District reported challenge as 





a major challenge 1 

Districts rated the challenge 

District challenge 

Percent 

Low Cl 

High Cl 

Percent 

Low Cl 

High Cl 

Insufficient funding to implement whole- 
school or turnaround intervention models 

65 

51.8 

79.0 

65 

52.2 

78.0 

Insufficient funding to make substantial 
changes to school day or year schedules 

57 

39.4 

73.8 

80 

72.8 

87.2 

Restrictions in rules and regulations regarding 
extent of autonomy that schools can be 
granted in terms of staffing or budgets 

48 

31.2 

64.0 

68 

56.1 

79.7 

Insufficient funding to support school-based 
experts (outside consultants, instructional 
specialists or coaches, mentors) 

45 

31.4 

59.4 

93 

90.1 

96.8 

Insufficient funding to support special 
programs for students and families 

41 

27.6 

55.4 

92 

88.9 

96.0 

Concerns or opposition from parents or 
community groups about closing or 
restructuring schools 

41 

23.0 

58.9 

50 

37.3 

63.7 

Restrictions in rules and regulations regarding 
extension of school days or years 

40 

22.6 

56.4 

68 

55.3 

79.9 

Insufficient help from local social services and 
other community-based organizations in 
providing services to students and their 
families 

38 

23.5 

52.6 

87 

77.2 

97.0 

Restrictions in rules and regulations regarding 
number of schools that can be closed, opened 
as charters or restructured in other ways 

33 

16.1 

49.1 

27 

18.2 

35.4 

Lack of district staff capacity or expertise to 
screen or provide guidance or advice about 
EMOsand CMOs 3 

31 

17.8 

44.2 

32 

22.6 

41.5 

Unwillingness of high-performing teachers to 
move to low-performing schools 

30 

22.8 

36.6 

38 

28.1 

47.3 

Lack of district staff capacity or expertise to 
provide guidance or advice concerning whole- 
school or turn around intervention models 

29 

12.6 

45.7 

66 

52.3 

80.3 

Lack of evidence about performance of CMOs 
or EMOs or other intervention experts 3 

22 

11.4 

31.9 

32 

23.3 

41.7 


(continued) 
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Table F-4. Confidence intervals for percentage of districts with low-performing schools that 

reported major challenges when supporting school restructuring and improvement: 
2011-12 (cont'd) 



District reported challenge as 





a major challenge 1 

Districts rated the challenge 

District challenge 

Percent 

Low Cl 

High Cl 

Percent 

Low Cl 

High Cl 

Lack of clear SEA guidance or support focused 
on adoption of whole-school reform models 

15 

10.2 

20.5 

54 

39.5 

69.0 

Lack of district staff capacity or expertise to 
train instructional specialists, coaches, lead 
teachers, or school-based professional 
development staff 

15 

7.8 

22.6 

94 

91.1 

97.0 

Current data systems make tracking the 
success of school improvement efforts difficult 

10 

5.1 

15.6 

96 

93.1 

97.9 

Lack of evidence about effectiveness of school 
improvement models 

9 

4.5 

14.1 

77 

63.8 

89.8 


1 The challenge could be rated as not a challenge, a minor challenge, or a major challenge. Districts that identified the 
challenge as not applicable and districts without low-performing schools were excluded from the denominator. 

2 Districts rating the challenge excludes districts that identified the challenge as not applicable, districts that did not respond to 
the question, and districts without low-performing schools. 

3 An EMO is an education management organization. A CMO is a charter management organization. 

Notes: The percentages in the table are cross-sectional estimates for the population of districts with low-performing schools 
that were operating in the 2010-11 school year. See appendix A for more information about the generalizability of the 
estimates. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table F-5. Results of significance tests comparing percentages of districts with low-performing 
schools reported major challenges when supporting school restructuring and 
improvement: 2011-12 


Ref. # 
1 
2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 
17 


Challenge description Percent 1 

Insufficient funding to implement whole-school or turnaround intervention models 65 

Insufficient funding to make substantial changes to school day or year schedules 57 

Restrictions in rules and regulations regarding extent of autonomy that schools can be 
granted in terms of staffing or budgets 

Insufficient funding to support school-based experts (outside consultants, instructional ^ 

specialists or coaches, mentors) 

Insufficient funding to support special programs for students and families 41 

Concerns or opposition from parents or community groups about closing or restructuring 
schools 


Restrictions in rules and regulations regarding extension of school days or years 40 

Insufficient help from local social services and other community-based organizations in 
providing services to students and their families 

Restrictions in rules and regulations regarding number of schools that can be closed, opened ^ 

as charters or restructured in other ways 

Lack of district staff capacity or expertise to screen or provide guidance or advice about 
EMOs and CMOs 1 


Unwillingness of high-performing teachers to move to low-performing schools 30 

Lack of district staff capacity or expertise to provide guidance or advice concerning whole- ^ 

school or turn around intervention models 

Lack of evidence about performance of CMOs or EMOs or other intervention experts 2 22 

Lack of clear SEA guidance or support focused on adoption of whole-school reform models 15 

Lack of district staff capacity or expertise to train instructional specialists, coaches, lead 
teachers, or school-based professional development staff 

Current data systems make tracking the success of school improvement efforts difficult 10 

Lack of evidence about effectiveness of school improvement models 9 


1 Percentage of districts with low-performing schools that rated the challenge as major. They are cross-sectional estimates for 
the population of districts operating in the 2010-11 school year. See appendix A for more information about the 
generalizability of the estimates. 

2 An EMO is an education management organization. A CMO is a charter management organization. 

(continued) 
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Table F-5. Results of significance tests comparing percentages of districts with low-performing 
schools reported major challenges when supporting school restructuring and 
improvement: 2011-12 (cont'd) 


Ref 

# 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

1 



* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

2 





* 


* 



* 

* 

* 

* 

* 

* 

* 

* 

3 








* 



* 

* 

* 

* 

* 

* 

* 

4 











* 

* 

* 

* 

* 

* 

* 

5 














* 

* 

* 

* 

6 












* 


* 

* 

* 

* 

7 














* 

* 

* 

* 

8 














* 

* 

* 

* 

9 
















* 

* 

10 














* 

* 

* 

* 

11 














* 

* 

* 

* 

12 

















* 

13 
















* 

* 

14 

















* 

15 
















* 


16 


















17 



















^Percentages are significantly different (p < 0.05). 

Table reads: As denoted by no asterisk in row 1, column 2, the percentage of districts that reported insufficient funding to 
implement whole-school or turnaround intervention models as a major challenge (65 percent) is not significantly different from 
the percentage that rated Insufficient funding to make substantial changes to school day or year schedules as a major challenge 
(57 percent). Use the reference number (Ref. N) to identify the challenge. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 
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Table F-6. Confidence intervals for percentage of low-performing schools that supported 
improvement: 2009-10, 2010-11, and 2011-12 




2009-10 



2010-11 



2011-12 




Low 

High 


Low 

High 


Low 

High 

Indicator description 

% 

Cl 

Cl 

% 

Cl 

Cl 

% 

Cl 

Cl 

School Implemented Programs to 
Encourage Family and Community 
Involvement 

88 

80.1 

95.3 

A 

95 

86.8 

100.0 

98 + 

96.8 

99.2 

School Implemented Programs to 
Address Students' Social and Emotional 
Needs 

81 

72.9 

88.7 

< 

<T> 

00 

81.7 

96.8 

91 + 

88.1 

94.1 

School Implemented Programs to Orient 
Parents to School Improvement Models 

69 

63.6 

74.2 

A 

80 

72.7 

88.0 

79 + 

73.9 

83.9 

School Modified Daily Schedule to 
Increase Instructional Time for 
Reading/English Language Arts or 
Mathematics 

58 

52.0 

63.6 

A 

72 

65.0 

79.8 

78 + 

72.9 

82.5 

School Used Outside School 
Improvement Experts as Part of School 
Restructuring or to Improve Instruction 

47 

41.3 

53.4 

44 

35.7 

52.3 

66 +,§ 

59.9 

71.5 


Nonfinancial Incentives to Encourage 
Teachers to Move to or Remain in the 
School 

22 

13.6 

31.0 

A 

24 

15.6 

32.9 

21 

13.3 

29.0 

School Day, Week, or Year Extended as 
Part of School Restructuring 

12 

8.7 

14.8 

10 

7.7 

12.3 

19 +,§ 

15.6 

22.4 

Substantial Portion of Teachers Replaced 
as Part of School Restructuring 

8 

1.5 

14.1 

4 

2.7 

5.1 

§ 

11 

7.3 

15.4 

Compensation Incentives for Teachers 
Who Move to Teach at the School 

5 

2.8 

7.3 

5 

2.9 

7.1 

5 

2.5 

6.9 

Staffing or Budgeting Autonomy 
Increased as Part of School Restructuring 

5 

2.6 

6.4 

7 

0.9 

12.7 

9 + 

6.3 

12.6 

Effective Teachers Reassigned to School 
as Part of School Restructuring 

4 

1.8 

5.4 

4 

2.5 

6.1 

5 

3.0 

6.0 


Percentage for 2010-11 is significantly different from percentage for 2009-10 (p < 0.05). 

^Percentage for 2011-12 is significantly different from percentage for 2009-10 (p < 0.05). 

§ Percentage for 2011-12 is significantly different from percentage for 2010-11 (p < 0.05). 

Notes: The percentages in the figure are cross-sectional estimates for the population of low-performing schools that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. In this 
figure, the denominator is the estimated number of low-performing schools. 

Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Surveys. Approved state applications for School 
Improvement grants for low-performing schools data. Retrieved December 2010 from http://www2.ed.gov/programs/sif/ . 
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Table F-7. Results of significance tests comparing percentages of low-performing schools that 
supported improvement: 2011-12 

Percent 

Ref. # Indicator description of schools 1 

1 School Implemented Programs to Encourage Family and Community Involvement 98 

2 School Implemented Programs to Address Students' Social and Emotional Needs 91 

3 School Implemented Programs to Orient Parents to School Improvement Models 79 

4 School Modified Daily Schedule to Increase Instructional Time for Reading/English 
Language Arts or Mathematics 

5 School Used Outside School Improvement Experts as Part of School Restructuring or to 

, , • 66 
Improve Instruction 

6 Nonfinancial Incentives to Encourage Teachers to Move to or Remain in the School 21 

7 School Day, Week, or Year Extended as Part of School Restructuring 19 

8 Substantial Portion of Teachers Replaced as Part of School Restructuring 11 

9 Staffing or Budgeting Autonomy Increased as Part of School Restructuring 9 

10 Compensation Incentives for Teachers Who Move to Teach at the School 5 

11 Effective Teachers Reassigned to School as Part of School Restructuring 5 

1 These are the percentage of low-performing schools that rated the challenge as major. They are cross-sectional estimates for 
the population of schools operating in the 2010-11 school year. See appendix A for more information about the 
generalizability of the estimates. 

(continued) 
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Table F-7. Results of significance tests comparing percentages of low-performing schools that 
supported improvement: 2011-12 (cont'd) 


Ref. #1 2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 * 

* 

* 

* 

* 

* 

* 

* 

* 

* 

2 

* 

* 

* 

* 

* 

* 

* 

* 

* 

3 



* 

* 

* 

* 

* 

* 

* 

4 



* 

* 

* 

* 

* 

* 

* 

5 




* 

* 

* 

* 

* 

* 

6 






* 

* 

* 

* 

7 






* 

* 

* 

* 

8 








* 

* 

9 








* 

* 

10 










11 











*Percentages are significantly different (p < 0.05). 

Table reads: As denoted by the asterisk in row 1, column 2, a significantly larger percentage of implemented programs to 
encourage family and community involvement (98 percent) than implemented programs to address students' social and 
emotional needs (91 percent). Use the reference number (Ref. #) to identify the indicator. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table F-8. Confidence intervals for percentage of low-performing schools that reported major 
challenges when working on school organization and improvement: 2011-12 



School reported 





challenge as a major 

School rated the 



challenge 



challenge 




Low 

High 


Low 

High 

School challenge 

% 

Cl 

Cl 

% 

Cl 

Cl 

Restrictions in rules and regulations on replacing less effective 
teachers 

49 

43.7 

55.2 

76 

68.5 

84.3 

Insufficient funding to make substantial changes to school day 
or year schedules 

45 

35.9 

53.6 

73 

65.5 

80.0 

Insufficient funding to support special programs for students 
and families 

37 

29.9 

45.0 

83 

76.4 

89.5 

Restrictions in rules and regulations on making substantial 
changes to school day or year schedules 

36 

29.6 

43.2 

75 

67.7 

82.1 

Insufficient funding to support school-based experts (outside 
consultants, instructional specialists or coaches, mentors) 

36 

28.8 

43.9 

83 

76.8 

90.1 

Insufficient funding to purchase technology for classroom use 

32 

26.1 

38.8 

82 

74.3 

89.3 

Insufficient help from local social services and other 
community-based organizations in providing services to 
students and their families 

22 

17.5 

25.9 

82 

74.2 

89.0 

Lack of clear district guidance or support focused on staffing 
or budgeting decisions made at the school level 

20 

15.5 

25.1 

77 

68.5 

85.6 

Lack of school staff capacity or expertise to effectively use 
technology to improve instruction 

19 

14.6 

23.4 

82 

74.6 

89.5 

Lack of clear district guidance or support focused on 
implementing a whole-school intervention or turn around 
model 

17 

11.7 

21.4 

69 

60.7 

77.1 

Concerns or opposition from parents or community groups 
about reform activities 

9 

4.8 

12.6 

75 

67.0 

83.3 


1 The challenge could be rated as not a challenge, as a minor challenge, or as a major challenge. Low-performing schools that 
identified the challenge as not applicable and schools that were not identified as low-performing were excluded from the 
denominator. 

2 Schools rating the challenge excludes schools that identified the challenge as not applicable, schools that did not respond to 
the question, and schools that were not low performing. 

Notes: The percentages in the table are cross-sectional estimates for the population of low-performing schools that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. 
Sources: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. Approved state applications for School Improvement grants. 
Retrieved December 2010 from http://www2.ed.gov/programs/sif/ . 
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Table F-9. Results of significance tests comparing percentages of low-performing schools that 
reported major challenges when working on school organization and improvement: 
2011-12 

Ref. # Challenge description Percent 1 

1 Restrictions in rules and regulations on replacing less effective teachers 49 

2 Insufficient funding to make substantial changes to school day or year schedules 45 

3 Insufficient funding to support special programs for students and families 37 

4 Restrictions in rules and regulations on making substantial changes to school day or year 
schedules 

5 Insufficient funding to support school-based experts (outside consultants, instructional 
specialists or coaches, mentors) 

6 Insufficient funding to purchase technology for classroom use 32 

7 Insufficient help from local social services and other community-based organizations in ^ 

providing services to students and their families 

8 Lack of clear district guidance or support focused on staffing or budgeting decisions made 
at the school level 

9 Lack of school staff capacity or expertise to effectively use technology to improve 
instruction 

10 Lack of clear district guidance or support focused on implementing a whole-school ^ 

intervention or turn around model 

11 Concerns or opposition from parents or community groups about reform activities 9 

1 These are the percentage of low-performing schools that rated the challenge as major. They are cross-sectional estimates for 

the population of schools operating in the 2010-11 school year. See appendix A for more information about the 

generalizability of the estimates. 

(continued) 


F-12 


Table F-9. Results of significance tests comparing percentages of low-performing schools that 
reported major challenges when working on school organization and improvement: 
2011-12 (cont'd) 


Ref. # 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 



* 

* 

* 

* 

* 

* 

* 

* 

* 

2 



* 

* 

* 

* 

* 

* 

* 

* 

* 

3 







* 

* 

* 

* 

* 

4 







* 

* 

* 

* 

* 

5 







* 

* 

* 

* 

* 

6 







* 

* 

* 

* 

* 

7 










* 

* 

8 










* 

* 

9 











* 

10 











* 

11 













*Percentages are significantly different (p < 0.05). 


Table reads: As denoted by no asterisk in row 1, column 2, the percentage of schools that reported restrictions in rules and 
regulations on replacing less effective teachers as a major challenge (49 percent) is not significantly different from the 
percentage that rated insufficient funding to make substantial changes to school day or year schedules as a major challenge (45 
percent). 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table G-l. Percentage of districts that implemented reforms in standards and assessments, data 
systems, and educator workforce development in 2009-10 and 2011-12, by number of 
indicators met 


Number of indicators met 

Percent 

2009-10 

2011-12 

12 

<1 

1 

11 

3 

2 

10 

3 

8 

9 

10 

7 

8 

14 

14 

7 

15 

14 

6 

15 

17 

5 

10 

15 

4 

16 

9 

3 

7 

6 

2 

2 

3 

1 

2 

2 

0 

3 

<1 


Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. The denominator for these percentages is the estimated number of districts with sufficient data (i.e., 
answered enough questions to calculate the indicator). Because the Common Core State Standards were not available until 
2010-11, for 2009-10 the number of indicators met includes the district's 2010-11 status for the four standards and 
assessments indicators. 
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Table G-2. 


Percentage of schools that implemented reforms in standards and assessments, data 
systems, and educator workforce development in 2009-10 and 2011-12, by number of 
indicators met 


Number of indicators met 

Percent 

2009-10 

2011-12 

11 

1 

1 

10 

2 

4 

9 

5 

8 

8 

10 

15 

7 

14 

21 

6 

19 

19 

5 

16 

14 

6 

13 

9 

7 

9 

5 

2 

4 

2 

1 

1 

<1 

0 

5 

3 


Notes: The percentages in the table are cross-sectional estimates for the population of schools that were operating in the 2010- 
11 school year. Because the Common Core State Standards were not available until 2010-11, for 2009-10 the number of 
indicators met includes the school's 2010-11 status for the three standards and assessments indicators. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Survey. 
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Table G-3. 

Confidence intervals for percentage of districts that implemented reforms in standards 
and assessments, data systems, and educator workforce development in 2011-12, by 
number of indicators met 

Number of indicators met Percent of all districts 

Low Cl High Cl 

12 

1 

0.5 

1.6 

11 

2 

1.3 

2.6 

10 

8 

2.8 

13.9 

9 

7 

5.1 

8.7 

8 

14 

10.4 

18.0 

7 

14 

11.5 

16.5 

6 

17 

11.7 

23.2 

5 

15 

9.0 

20.6 

4 

9 

5.6 

13.0 

3 

6 

2.5 

8.8 

2 

3 

1.8 

5.1 

1 

2 

1.2 

3.6 

0 

0.49 

0 

1.0 

Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year. See appendix A for more information about the generalizability of the estimates. 


Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 


Table G-4. 

Confidence intervals for percentage of schools that implemented reforms in standards 
and assessments, data systems, and educator workforce development in 2011-12 


Number of indicators met Percent of all schools 

Low Cl High Cl 

11 

1 

0.5 

1.5 

10 

4 

2.8 

4.8 

9 

8 

7.0 

9.9 

8 

15 

12.4 

17.2 

7 

21 

18.0 

23.0 

6 

19 

16.3 

21.1 

5 

14 

12.0 

16.3 

4 

9 

6.5 

10.5 

3 

5 

1.8 

8.6 

2 

2 

0.7 

2.3 

1 

0.46 

0.2 

0.7 

0 

3 

1.9 

4.0 


Notes: The percentages in the table are cross-sectional estimates for the population of schools that were operating in the 2010- 
11 school year. See appendix A for more information about the generalizability of the estimates. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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Table G-5. Percentage of low-performing schools that supported reforms in school improvement in 
2009-10 and 2011-12, by number of indicators met 


Number of indicators met 

Percent 

2009-10 

2011-12 

11 

0 

<1 

10 

<1 

1 

9 

<1 

<1 

8 

2 

2 

7 

4 

5 

6 

8 

12 

5 

21 

31 

4 

27 

26 

3 

18 

10 

2 

7 

4 

1 

7 

1 

0 

7 

8 


Notes: The percentages in the table are cross-sectional estimates for the population of low-performing schools that were 
operating in the 2010-11 school year. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2011 and Spring 2012 School Survey. 
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Table G-6. 

Confidence intervals for percentage of districts with low-performing schools that 
implemented school improvement reforms in 2011-12 


Percent of districts with 
Number of indicators met low performing schools 

Low Cl 

High Cl 

9 

0.08 

0.02 

0.1 

8 

1 

0 

1.2 

7 

3 

1.9 

3.6 

6 

2 

1.3 

3.4 

5 

9 

2.8 

14.4 

4 

16 

5.1 

27.5 

3 

17 

11.4 

23.2 

2 

22 

8.5 

35.8 

1 

13 

8.0 

18.1 

0 

17 

2.8 

31.0 

Notes: The percentages in the table are cross-sectional estimates for the population of districts that were operating in the 
2010-11 school year and have low-performing schools. See appendix A for more information about the generalizability of the 
estimates. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 District Survey. 


Table G-7. 

Confidence intervals for percentage of low-performing schools that supported reforms in 
school improvement in 2011-12 

Number of indicators met Percent of low performing schools 

Low Cl 

High Cl 

11 

0.01 

0 

0.03 

10 

1 

0 

1.4 

9 

0.38 

0 

0.79 

8 

2 

1.2 

3.0 

7 

5 

3.6 

6.9 

6 

12 

8.4 

16.0 

5 

31 

24.8 

36.7 

4 

26 

17.0 

34.1 

3 

10 

7.1 

13.9 

2 

4 

1.9 

5.8 

1 

1 

0.2 

2.4 

0 

8 

2.1 

13.1 


Notes: The percentages in the table are cross-sectional estimates for the population of low-performing schools that were 
operating in the 2010-11 school year. See appendix A for more information about the generalizability of the estimates. 

Source: U.S. Department of Education, Institute of Education Sciences, Charting the Progress of Education Reform: An 
Evaluation of the Recovery Act's Role: Spring 2012 School Survey. 
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